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Foreword 



On June 23, 1988, the Nelson A. Rockefeller Institute of Government and Empire State 
College jointly sponsored a conference on assessing performance in higher education. The 
conference was designed for top policymakers in New York State and more than 125 
executives from business, government, and higher education were invited. Individuals 
attending the conference came from several New York State agencies, legislative committees, 
business and industrial corporations, the Business Council, and unions as well as ail sectors of 
higher education in the state, including the State University of New York's university centers, 
university colleges, and community colleges, the City University of New York, private colleges 
and universities, and the State Education Department. 

The conference was organized to bring together experts on assessment and accountability 
issues who presented national, state, and institutional perspectives and working models to the 
audience. This Special Report places the conference presentations in a larger context of 
undergraduate reform, the demands for accountability, existing models of excellence, and new 
approaches to assessment. With a brief conclusion and extensive bibliography, the entire 
report is intended to provide interested policymakers with a more comprehensive discussion of 
assessment and accountability issues involved in the undergraduate reform movement. The 
report is also intended to highlight effective models and practices that the state of New York 
may consider as it formulates educational policies in this area. 

The authors would like to thank the Norstar Bank for providing an endowment fund to 
the Rockefeller Institute for an annual conference on excellence in education. The expertise 
and efforts of many people Inade the 1988 conference a successful experience. The authors 
would like to express their thanks to those on the program who shared their views and 
experiences on accountability and assessment issues: James W. Hall, President, Empire State 
College; Thea Hoeth, Special Assistant to the Governor for Management and Productivity, 
State of New York; Joseph C. Burke, Provost, State University of New York; Carol M. Boyer, 
Senior Policy Analyst, Education Commission of the States; Edward B. Fiske, Education 
Editor, The New York Times; Homer S. Fisher, Executive Vice Chancellor for Business, 
Planning and Finance, Tl j University of Tennessee at Knoxville; Trudy W. Banta, Research 
Professor and Director, Assessment Resource Center, The University of Tennessee at 
Knoxville; D. W. Farmer, Vice President for Academic Affairs, King's College, Pennsylvania; 
E. Thomas Moran, Assistant Vice President for Academic Affairs, State University College at 
Pittsburgh. 

Special thanks go to the staff of the Nelson A. Rockefeller Institute of Government for 
their cosponsorship of the conference, their assistance and support in planning the conference, 
and for assisting in the preparation of the Special Report. We would like to especially 
recognize David Andersen, Director of the Rockefeller Institute, Cathy Stone, Director of 
Development and Special Projects, Robin Mandelkorn, Intern, and Susan Lenz, Production 
Manager. 

Finally, the authors wish to thank James W. Hall, President of Empire State College, 
who endorsed and fully supported the idea of this conference. We also want to thank Judy 
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Remington who transcribed and typed all the conUrence speeches and who prepared the 
various drafts and final version of the text for this Special Report. 
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I. A CONTEXT FOR CHANGE: 
The Reform Movement in Higher Education 
Judith M. Holtan, ?h,D. 



In recent years, government policymakers and educators have been engaged in a 
dialogue concerning the quality of undergraduate education. The outgrowth of this concern has 
been the current educational reform movement that includes improved institutional 
accountability and outcomes assessment* The underlying assumption in the push for improved 
accountability through assessing student learning is that the traditional methods of assessing 
individual student achievement, and program and institutional effectiveness, are inadequate. 
Further, it is assumed that improving the measurement of learning outcomes will, necessarily, 
lead to institutional self-assessment and greater effectiveness in the fulfillment of institutional 
missions* But assessment has its critics — and its controversies. For example, neither group — 
state policymakers or educators — has agreed upon the meaning of "quality" in higher 
education; neither group has determined what should be the outcomes of an undergraduate 
education; and neither group seems clear about how one best assesses those outcomes. 
Nevertheless, a number of states have already endorsed the concept of assessment to achieve 
exceUence in higher education and to demonstrate accountability, and have instituted 
programs of surveying and testing students. Perhaps there will never be consensus on these 
issues, but we now have sufficient experience in assessment to frame the dialogue better and 
to move the discussion in more fruitful directions. 

Because this is an issue that is gaining in importance nationally, and because the 
dialogue and the discussion are critical to the future of undergraduate education, we developed 
the conference on "Assessing Performance in Higher Education" and include the presentations 
in this publication. The conference brought together a number of policymakers from 
government, business, and higher education to discuss with the experts the twin issues of 
accountability and assessment from a national and a state perspective. The following is 
designed to provide an introduction to the conference presentations by setting forth the major 
issues and the context from which the current reform movement in higher education emerged. 



The National Reports 

Assessment, incorporating new approaches to defining student learning, is the major 
part of the current reform in undergraduate education as the higher education community 
responds to the challenges issued in the spate of national surveys and reports released during 
the mid-eighties that revealed a decline in overall academic achievement nationally. The 
reports also revealed a growing sense by leaders from business and industry, the professions, 
and the government that recent college graduates were not meeting their expectations in 
preparation for the workplace. As a result, political leaders, particularly at the state level, 
began to request better measures of accountability from colleges and universities to justify the 
funding of higher education. Rather than relying upon the traditional accreditation standards — 
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overly dependent upon resource and process measures — state governments now want 
information on the outcomes of students, programs, and institutions. But more importantly, 
their concern is directed towards assurance that higher education in America provides quality 
education that will meet the demands of the twenty-first century. 

In 1983, the National Commission on Excellence in Education issued its report A Nation 
at Risk: The Imperative for Educational Reform that warned of a "rising tide of mediocrity" 
unless academic standards were raised in our nation's secondary schools. The deficiencies 
found in high school education were linked to the relaxed standards for the baccalaureate 
degree over the preceding fifteen years. Among the indicators of "risk" cited in the report 
were: the steady decline in the Scholastic Aptitude Test (SAT) scores; the lowered admissions 
standards of the more "selective" colleges and universities; and weakened teacher preparation 
through a curriculum that stressed methods courses rather than coursework in the subject areas 
and in general education. A report from the College Board in 1984, with data from their 
Admissions Testing Program between 1972 and 1983, confirmed the decline in academic 
performance among those taking the SATs. In 1972, 56 percent more students scored above 
600 than in 1984 and of those scoring above 650, there were 73 percent more in 1972. 

Another national report released in 1984, Involvement in Learning: Realizing the 
Potential of American Higher Education from the National Institute of Education, called for a 
renewed emphasis on undergraduate teaching and proposed that new priorities be set in 
American higher education, stating: 

The higher education system sets the tone for the whole of American education 
and determines the aspirations of students at all levels. Americans respect learned 
men and women who have mastered the highest reaches of knowledge. Thus, if 
American higher education settles for less than the best — if it allows the chase for 
academic credentials to supersede the pursuit of learning — all levels of education 
will suffer (p. 14). ^ 

The report cited recent studies that showed a "disturbing trend" in the scores of the subject 
area tests of the Graduate Record Examination (GRE), in which student performance 
declined in eleven of the fifteen major subject areas from 1964 to 1982. The report argues 
strong xor the development of assessment programs that will measure student learning 
outcomes and provide information for curriculum planning and program planning, 
recommending that institutions "make a conscientious effort to acquire and to use better 
information about student learning, the effects of courses, and the impact of the programs" (p. 
21). A follow-up study to the Commission's report refined the data and included the test 
scores from the professional school entrance examinations, such as the Law School 
Admissions Test (LSAT) and the Graduate Management \dmissions Test (GMAT). Since 
over a half-million of these tests are taken every year, a decline in the scores over several 
years could suggest a deterioration in the quality of undergraduate preparation. The author of 
the report, however, cautions: the scores of the tests do net tell us about the quality of 
American higher education, they only reflect the changing quality of student learning 
(Adelman, 1984). 

State leaders have been among the most vocal critics of higher education. In the 
National Governors' Association repoit, Time for Results: The Governors' 1991 Report on 
Education (1986), the Task Force on College Quality included the following statement 
outlining the need for greater accountability through assessment programs: 



America's institutions of higher education stand as a gateway to opportunity, 
enterprise, and individual growth. Substantial public and private funds support 
these institutions. Public policymakers, taxpayers, students, and parents should 
expect colleges and universities to fulfill their promises. To assure accountability, 
post-secondary institutions must assess student learning and ability, program 
effectiveness, and institutional accomplishment of mission (p. 159). 

The current interest in assessing learning outcomes by state leaders is linked to more than 
financial accountability — there is the recognition that excellence in education bears upon 
future economic growth and prosperity. In the mid-eighties two other national reports 
appeared that issued much the same challenge for higher education. Noting that with the 
growth of professional studies and increasingly narrow specialization, student interest in 
general education had waned, the reports, To Reclaim A Legacy by William J. Bennett and 
Integrity in the College Curriculum: A Report to the Academic Community from the 
Association of American Colleges, called for a renewed emphasis on liberal learning. 

The circle of debate that currently surrounds the issue of quality in education has 
widened to embrace the professoriate. Two critiques of American education have made the 
best-seller lists the past year. Allan Bloom's The Closing of the American Mind and E. D. 
Hirsch's Cultural Literacy address our declining expectations of what the educated person 
should be. The former is directed specifically at higher education, while Cultural Literacy 
takes on the average high school graduate in America. Each book, in its own way, prescribes 
a return to the traditional values of the liberal arts curriculum. But the point to be made here 
is, they have drawn a new audience to their arguments — the interest in educational reform has 
moved beyond the policymakers and the educators to appeal to an American public that 
suspects it has been settling for something less than it deserves in its educational system. 

Since regional accreditation has been the primary means for the academic community to 
ensure quality, the reform movement in higher education has meant that the process of 
accreditation has also come under attack. Regional accreditation is higher education's 
certification of quality. Through a process of self-study and evaluation by a team of external 
peer reviewers, each institution that meets the standards set by its regional association 
receives accreditation. The emphasis has been placed on an institution's resources (inputs). 
Thus the more easily quantifiable information, such as student-faculty ratios, numbers of 
volumes in the library, faculty with doctorates, etc., becomes the important criteria in judging 
the quality of an institution. Resources for learning are important. The case, however, rests on 
whether the resources are used in such a way that the institution's mission is achieved. The 
quality of the academic program and the effectiveness of teexhing undergraduates are too 
often inferences made with little substantive evidence to back them. After pointing out the 
accrediting agencies' inability to ac? 'ress the decline in academic standards, the report, Time 
for Results, states among its recommendations: "The higher education accrediting community 
should require colleges and univexdties to collect and utilize information about undergraduate 
outcomes. Demonstrated levels of student learning and performance should be considered in 
granting institutional accreditation" (p. 163). 

The Commission on Higher Education of the Middle States Association of Colleges and 
Schools, in its standards for accreditation (1982), includes this statement: "Efforts should be 
undertaken to assess student achievement in general as well as in specialized areas of the 
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curriculum on an inclusive as well as course-by-course basis, and when possible in 
comparison with student achievement elsewhere. Institutions should make frequent appraisals 
of the records of their graduates in education beyond the two- or four-year college level, 
survey alumni opinions, reports from their employers" (p. 11). Nevertheless, input measures 
still remain the important criteria in judging the quality of an institution. And regional 
accreditation still remains the primary means for the academic community to ensure quality, 
although its critics have not failed to notice the implications of its weaknesses. As Glen 
Dumke, writing in The Chronicle of Higher Education (January 1986), noted: "If accreditation 
is to recover the confidence it once enjoyed, it must do something about degree requirements. 
There was a time in this country when a bachelor's degree connoted certain abilities, including 
some competence in written and spoken English, some understanding of the cultural amenities, 
a knowledge of history, and some basic skills in mathematics and science." Strong words. But 
the writer's point goes back to the concern for the variation in quality from institution to 
institution and the insufficient attention that has been paid to the learning outcomes of the 
institution's graduates. 

The American higher education system is a vast network made up of over three 
thousand institutions whose size and mission range from small technical colleges to vast 
research universities. Nearly half of the institutions are private — sectarian and nonsectarian — 
colleges and universities. The remaining colleges and universities belong to the public sector 
with most of them controlled by independent governing boards. American higher education is 
unique because of its extreme diversity and its decentralized nature. Its institutions of higher 
education arrange themselves in a hierarchy through a process of selectivity. At the top of the 
hierarchy are the most prestigious research universities able to attract students with the highest 
admissions test scores and faculty with the best national reputations. The lower levels of the 
hierarchy are made up of two-year colleges with open admissions policies. Our notions of 
quality in higher education have been shaped by the reputations of a few institutions at the top 
(Astin, 1985). We may have to change our notions. As the debate about excellence in 
undergraduate education has deepened, through greater concern for teaching and student 
development, the institutions that have taken a leadership role have not been among those at 
the top of the hierarchy. Three of these institutions immediately come to mind — Alverno 
College, Northeast Missouri State University, and King's College in Pennsylvania. They have 
successful assessment programs that have been in place for a number of y?ars. Each institution 
uses assessment in a slightly different way to support their educational goals; all, however, can 
serve as an example of what can occur through the collaborative efforts of the faculty, the 
students, and the administration when common goals of educational achievement are set forth. 
The information that assessment generates can be used to improve program review, strategic 
planning, and the budgetary process. Assessment is a key io providing greater accountability 
to internal and external audiences. 



The Demand for Accountability 

The nation has an important stake in the quality of its higher education. The increased 
interest in greater accountability by colleges and universities is linked to the public's awareness 
of the role that post-secondary education plays in shaping the social, economic* and political 
environment of America. The issues that have forced the accountability debate can be 



organized into four major themes: economic competitiveness, the values of a democratic 
society, fiscal accountability, and consumer protection. In the past, higher education avoided 
controversy because the public allowed it to monitor its own standards and performance 
through the accreditation process. But the immunity from this kind of public scrutiny began to 
change with the reform reports of the mid-eighties as they called attention to the role that 
education must play in our future well-'jeing. 



The Competitiveness Theme 

The foremost of the issues concerns Amerxa's weakened ability to meet increased 
competition from abroad unless we develop a labor force capable of generating the knowledge 
and the technologies required for sustained future economic growth. In 1986 executives from 
industry, labor, and higher education formed the Council on Competitiveness, designed to 
improve America's competitiveness in the world market. The Council developed a 
"Competitiveness Index" to chart U.S. performance against the seven other "summit" 
countries using 1972 as the base year. The index is set at 100 for that year; it includes figures 
on the standard of living, trade, productivity, and investment in future competitiveness. Their 
first report shows that the U.S. has not equalled the other countries in any of the four 
categories. Education spending is included in the index as part of the category "investment in 
future competitiveness" along with spending for nondefense research and development, and 
new plants and equipment. In 1986 the U.S. expenditure on education was 6.8 percent of the 
Gross Domestic Product (G.D.P.) compared to an average 3.48 percent tor the other 
countries; in 1972 U.S. spending for education was 7.2 percent of the G.D.P. , compared to 
their 3.72 percent. That our investment in education has remained nearly double that of the 
other countries while our competitiveness has declined, suggests that either education 
correlates rather weakly with economic growth and prosperity, or, as the project director of 
the study, Daniel F. Burton, Jr., states: "On paper, the U.S. is spending a lot on education. 
Whether or not we're getting a big enough bang for our buck is another question" (Education 
Week, June 15, 1988). The competitiveness index accounts for the country as a whole, but 
more interesting in view of the current push for educational reform, is the role that state 
policymaker hp\e played in what was previously considered a national issue. For the past two 
decades the U.S. has not enjoyed a period of sustained growth shared equally among most of 
its member states. With state policy-making becoming more sophisticated, in part the result of 
the reduced role of the federal government encouraged by Republican administrators, the 
states are becoming increasingly competitive among themselves for domestic and foreign 
investment. And a good part of their strategy Tor competitiveness has hinged on improving 
their states' educational systems to attract business and industry with a educated workforce. 

In a recent study that analyzed the role of education with respect to the workplace, 
Henry Levin and Russell Rumberger argue persuasively that technological skills are less 
important than "technological literacy" — knowledge of how technology shapes our lives in the 
workplace and outside — in meeting the challenge of the future. Using evidence from a number 
of studies (their own and others), they have concluded that the new technologies themselves 
will not have a profound effect in upgrading the education and skill requirements of most jobs. 
They stress, instead, the possibilities associated with the new technologies that can be 
incorporated into the organization of work to create jobs that are challenging and meaningful 
to the individual worker. 
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In our view, education not only responds to the needs of the workplace, but also 
has the power to shape them. Consequently, schools and universities should not 
simply provide the education and the training that educators and policymakers 
think future jobs will require, but should provide the education and the training 
that will help students — as workers, employers, and government officials — to shape 
the requirements of work. This includes helping to determine how work is 
organized and technology employed to provide favorable impacts on jobs and skill 
requirements. It includes the provision of better decision-making skills among 
workers so that they can use the data provided by an information-rich 
environment to make better choices regarding the use and allocation of productive 
resources. This view also suggests that the best preparation for the future is a 
general rather than a specific job-focused education, one with a strong foundation 
in the liberal arts (Levin and Rumberger, 1987, p. 350). 

Derek Bok, in Higher Learning (1986), has suggested that with the rapid increase in new 
knowledge students must be taught "habits of continuous learning" rather than a fixed body of 
knowledge. Thus analytical skills, techniques of problem solving, and the capacity for 
innovative and creative thought typically developed in the undergraduate years must be 
reaffirmed by the higher education community as it responds to its critics. King's College in 
the late 1970s, anticipating the reform in higher education, iaunched a long-range campaign of 
planning for excellence based upon the question: "What is the proper definition of excellence 
in higher education for students who will be living and working in the 21st century?" (Farmer, 
1988; "The King's College Experience," p. 64). Their response to the question was the 
development of an "outcomes-oriented" curriculum that emphasized a number of essential 
competencies within the goals t,f liberal learning agreed on by the faculty. Among them are 
such habits of learning as critical thinking, creative thinking and problem-solving strategies, 
library and information technologies, and quantitative analysis. 



The Values of a Democratic Society 

The second concern for maintaining quality in our colleges -<nd universities is based upon 
the recognition that higher education plays a significant part in training citizens for effective 
leadership and participation in our democratic society. Our higher education institutions 
perpetuate — through teaching, research, and public service — the democratic ideal by the 
transmission of our cultural heritage and the principles upon which our form of government 
was based. This goes back to the notion of liberal learning. But historical consciousness of the 
founding of the nation is aot enough. Higher learning should :*Iso develop a compassion for the 
diverse groups that create a pluralistic society, and a sensitivity to those continuities that 
regard the individual, yet bind us together as one nation. Our colleges and universities must 
also bear the social responsibility of providing access to the quality education that ensures 
citizens their full participation in society. As Joseph Burke pointed out in his presentation to 
the conference: 

[T]he growing demands of learning and life in a society based increasingly on 
knowledge and information require that a growing proportion of Americans benefit 
from higher education .... Student bodies on most campuses will become ever 
more diverse — more part-time, more minority, more mature, more handicapped. 
The residential college in a rural setting filled with recent high school graduates 
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Tvith good educational preparation and clear career goals has become the 
exception rather than the rule. Yet the conventional view of institutional excellence 
in higher education lingers on though the conditions that fostered it have long since 
gone ("Academic Assessment: The Right Thing for the Right Reason," p. 32). 

The current movement for accountability through the assessment of the outcomes of higher 
education is intended that students from all socioeconomic backgrounds have access to a 
quality undergraduate experience. 



Fiscal Accountability 

The third aspect of accountability involves accounting for the financial investment the 
public has made in higher education. Higher education in 1987-88 made up 2.8 percent of the 
Gross National Product, absorbing about $124 billion in resources. In the previous academic 
yeai, the average expenditure per full-time student from education and general funds was 
$8,924. This represented an increase of 24 percent from 1981-82 (Finn, 1988). The assessment 
movement received its major impulse from state governments when governors and legislators 
started to mandate such programs to provide greater accountability to the public for its 
investment in higher education. It seems fair to say, with the recent trend towards greater 
accountability among the other agencies of the states, that the reasoning simply extended itself 
into the higher education sector as well. Thea Hoeth discusses New York State's efforts in this 
regard in her presentation to the conference; the State Education Department, SUNY, and 
CUNY are included among fifteen state agencies that are undergoing a comprehensive 
program evaluation as part of an accountability effort by the administration of Governor 
Cuomo (see "Accountability for New York State Agencies/' p. 26). 

There have always been certain required reporting functions to provide oversight for 
public higher education, such as financial accounting and resource allocation information, 
information concerning physical assets, and enrollment data. Today's emphasis on 
accountability is different, for it stresses student outcomes as well as information concerning 
an institution's effectiveness in meeting the goals of its mission. The state's involvement in 
higher education is defensible not only t ~ause of its funding support, but also because it must 
ensure that all of its citizens are provided equal access to education. Thus state authority may 
also extend to private institutions as part of maintaining the higher education asset as a whole. 
Private institutions become accountable, too, when they share in state-funded financial aid 
programs for their students (Ewell, 1987). As the costs of maintaining their higher education 
systems have risen, state governments have become increasingly sensitive to the need for 
justifying the expenditure in performance terms. 



Consumer Protection 

Finally, there is an aspect of accountability that is linked to "consumer protection." 
Students as consumers of a service should be assured that the results of their financial 
investment and expenditure in time will match at least some of the claims made by the 
colleges and universities that recruited them. When students and their parents consider the 
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many catalogs and glossy recruitment brochures, and decide where they will invest the twenty 
to sixty thousand dollars it costs for an undergraduate education, they need to feel the 
institution will stand behind its assertions of quality* If an institution has promised career 
preparation and job placement in its recruiting activities, there should be sound information 
about those who have already graduated to back up its claims. It has been estimated that last 
year's graduates received their baccalaureate degrees with an average debt of $8,000. Students 
cc nsidering a major investment in education should be provided better information about the 
institution's past record in terms of student achievement and student development so they can 
make informed choices. Information about learning outcomes and the surveys of graduates 
lends credibility to college catalogs. 

The pressure for public accountability that goes beyond the simple reporting functions 
previously expected has created the most controversy within the recent movement tovards 
education reform. Aside from opposition to outside interference and on other general 
philosophic grounds, there are two important concerns about state-mandated assessment. 
There is the fear that assessment information will be publicly distributed to invite unfair 
comparisons among the institutions, with little regard for the specific mission of each 
institution. And the second concern, of particular interest to faculties, is the fear that 
mandated competency testing will shift the curriculum towards those areas in which students 
can easily be tested rather than maintaining a curriculum in support of a particular academic 
program- of study deemed important by the faculty. Despite assurances by state officials that 
faculties and administrators will bear primary responsibility for the development of an 
appropriate assessment program on their campuses, the fear remains that the tests will 
determine the teaching. Peter Ewell of the National Center for Higher Education Management 
Systems (NCHEMS) has noted the distinction between assessment to provide accountability to 
an external constituency and cessment as a part of ii/^rnal planning for institutional 
improvement. He cautions: "Regardless of its domain, the concept of accountability requires 
not only performance itself, but also requires public presentation of evidence that requisite 
performance has been achieved. Accountability information must be of a particular kind — 
generic, summative, and easily comprehended by lay audiences. The use of information for 
improvement requires a quite different kind of communication" (Ewell, 1987, p. 12). Among 
the issues he raises are the limitations inherent in statistical reporting — the most common form 
of presenting accountability information — and the problems associated with communicating 
negative information to external constituencies. Assessment information used internally for 
institutional improvement can be more qualitative and can be used as a feedback process that 
allows the students, the faculty, and the adi^inistration the opportunity to make informed 
decisions concerning academic achievement, the curriculum, and the allocation of resources. 
The role that assessment can play in strategic planning is to provide constant information that 
can be analyzed for defining the curricular objectives and institutional goals for the future. For 
the controversies that surround assessment as part of accountability should not obscure th<* 
place of assessment in the general trend towards educational reform — however one defines ihe 
means, academic improvement should be the intended goal. 



n. MODELS OF EXCELLENCE AND APPROACHES TO ASSESSMENT 



Timothy Lehmann 
Assistant Vice President for Research and Evaluation 
Empire State College 
Saratoga Springs, New York 



Models of Excellence 

The reform movement and the series of reform reports, commission studies, and national 
policy declarations just reviewed call for redefining undergraduate education in ways to meet 
the challenges of the 21st century. In the post-World War II period, American higher 
education has improved access so that more than twelve million students are enrolled; has 
expanded educational opportunities to all social classes, ethnic groups, and regions of the 
nation; and has come close to realizing the democratic dream of equality, whereby a college 
education is within the grasp of nearly everyone. No other nation in the world has achieved 
such a remarkable record of providing education for all. 

Yet the reform reports call into question the performance of this huge enterprise. Ever 
since A Nation at Risk was published in 1983, new calls for "undergraduate reform," "higher 
performance," "improved quality," and "increased accountability" have surged forward in 
state after state. Some state legislatures and governors have now mandated that colleges and 
universities develop assessment plans and "test" students at entry and subsequent points in 
their college careers. Several colleges themselves have pioneered in the past decade in 
establishing comprehensive assessment programs that demonstrate improved rk °rformance by 
students. 

To gain a sense of the magnitude and scope of the undergraduate reform movement, the 
Education Commission of the States (ECS) conducted nationwide surveys in 1986 and 1987. In 
1986, ECS found that all 50 states had initiatives underway to improve undergraduate 
education. The range of initiatives covered such topics as admission standards, high 
school/college transition, "value-added" assessment of student outcomes, alternative resource 
allocation and funding mechanisms, special incentive funding for undergraduate education, 
and blue-ribbon commissions on higher education within a state (Boyer and McGuinness, 
1986, p. 3). The 1987 survey revealed that two-thirds of the states now had formal initiatives 
called assessment but state authorities, except in a few cases, left to the institutions the design 
and/conduct of assessment strategies. As the survey notes, "governors and legislators have 
placed the quality of undergraduate education and student learning squarely on the state 
agenda. The state boards aim to keep it there" (Boyer, et aL, 1987, p. 9). Again, 'he range of 
activities state boards of higher education are pursuing is an impressive list: mandated 
statewide testing programs; testing for teacher education; early intervention programs; 
encouraging institutional action; assessment within existing statewide mechanisms; and 
statewide monitoring of other outcomes. The most recent update on the states can be found in 
Carol Boyei^s address to the Conference, beginning on page 36. 
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Another illustration of the scope and magnitude of the reform movement has been the 
activities of the American Association for Higher Education (AAHE). At its past three 
National Conferences, AAHE has established "an assessment action community" where those 
interested individuals could mest and discuss topics of mutual concern. With support from the 
Fund for the Improvement of P ost-Secondary Education (FIPSE), AAHE has also launched a 
yearly National Conference on Assessment and established an Assessment Forum. The third 
such conference was held in June 1988 ai:d was attended by almost 1,000 people, representing 
hundreds of colleges and universities. The conference addressed such topics as assessing 
general education outcomes, public policy and institutional practice, students, methods and 
instruments, and assessment impact and results. These illustrations and surveys bring us 
naturally to the section of this report which focuses on * >e definitions of excellence and 
different assessment strategies in use. 

Federal and state policymakers as well as higher education leaders have not yet agreed 
upon common definitions of quality, or what constitutes the core of an undergraduate 
education, or how best to demonstrate the performance of individual students and the 
effectiveness of entire institutions. In an effort to provide some common grounds for discussion 
and to reduce some of the confusion and controversy surrounding different ideas of quality or 
excellence, one insightful leader of the reform movement has set forth four models of 
excellence that have served to justify the development of higher education. Astin offers a 
critical appraisal of the strengths and weaknesses of each, "calling for a major redefinition of 
undergraduate education. Astin argues that the four approaches to excellence — the 
reputational, the resource, the content, and the outcome or talent development — are not 
"necessarily consistent with the educational missions of most institutions, that they interfere 
with our efforts to expand educational opportunity, and their use does not promote greater 
excellence in the system as a whole" (Astin, 1985, p. ix). Let us examine each of these, models 
of excellence so that we can better see why undergraduate education needs a new model to 
sustain education as we near the 21st century. 



The Reputationai Model of Excellence 

Survey after survey show the same set of colleges and universities at the top of the higher 
education enterprise. Excellence is the perceived view of a college's program. Common 
folklore underlying reputations seems stable and resistant to change, as surveys over the years 
show. Quality ratings reveal a halo effect with highly rated departments clustered in the same 
college. This approach is criticized as being subjective and not focusing directly on student 
learning and development. Only a limited number of colleges are "best" and opportunities for 
less-well-prepared students are limited. The reputational view stresses what others think of 
your program. 



" Tie Resource Model of Excellence 



Excellence is bringing together the right resources (students, f acuity, physical facilities, 
financing) and then student learning and development occur. This approach assumes the more 
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resources there are, the better the learning is. The focus is on such measures of quality as low 
student/faculty ratios; high student test scores; high faculty publication rates; low average class 
size; large numbers of library books; and high endowments. High selectivity tends to reduce 
social diver§ity of the student population. Critics argue this approach does not focus directly 
on student learning and it overemphasizes input variables. It also focuses energy on 
accumulating resources rather than analyzing their effective use. It also assumes only a few 
students can learn and succeed (meritocratic view)* The resources view stresses what the 
college nas. 



The Content Model of Excellence 

Excellence is what is taught, usually defined as the liberal arts and sciences* The most 
prestigious colleges stress the liberal arts. This approach favors the assertive, well-prepared 
student. It assumes there is faculty agreement about a core curriculum of desirable knowledge. 
Critics argue that there is little research evidence to support the superiority of the liberal arts 
curriculum and faculty responsibility ends with coverage of content. Student learning depends 
on exposure to the right subjects and students tend to take a passive role in learning. This 
approach encourages specialization and fragmentation of faculty interests and equates 
scholarly expertise with pedagogical ability. The content view stresses what faculty teach and 
present to students. 



The Talent— Development Model of Excellence 

Excellence is what a college does to develop the talent of its students — from time of 
entry to exit and beyond. The outcomes or talent-development approach assumes all students 
can learn and "succeed*" It stresses assessment of actual student learning, the development of 
multiple skills and the evaluation of the "whole student." This approach assumes the student is 
an active learner, meeting and exceeding the minimal performance standards. Critics argue 
this approach is too complex and lacks appropriate measures to determine the extent and 
level of student learning. Often, it reduces the creative aspects of learning to a lowest common 
denominator — test score results and simple cognitive skills. To fully implement this approach 
may require substantial retraining of faculty and administrators. The outcomes approach 
stresses what students are able to do as a result of their collegiate experience. (These brief 
statements on the models were adapted from Astin, 1985*) 



Approaches to Assessment 

Assessment strategies today have taken on much broader meanings within the context of 
evolving and different models of excellence. In an effort to illuminate several assessment 
strategies, especially those which embrace the talent development model of excellence, this 
Special Report describes eight specific approaches to assessment in higher education. For each 
approach, we shall discuss the basic assumptions, strengths and weaknesses, pioneering states 
and colleges where developed intensively, typical instruments used, and major results to date. 
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In practice, each approach is rarely implemented in its ideal form; rather, colleges tend to 
draw upon a mixture of methods, strategies, and instruments to improve student learning. The 
author is indebted to Theodore Marchese (1987) for a stimulating article using some of these 
approaches,which have been extended and enlarged upon here so that the reader may obtain a 
fuller sense of each approach. 



Assessment as Standardized Testing 

Standardized testing is perceived by many to be the equivalent of assessment largely 
because standardized testing has become recognized as a crucial achievement of measurement 
psychologists. The dominant image held by many is a room with hundreds of students taking a 
Regents exam or an SAT or a GRE paper-and-pencil, multiple-choice test using a machine- 
scored booklet of questions. The basic idea of the test is to measure a sample of the student's 
behavior, under controlled conditions, so that performance in a larger arena of behavior can 
be inferred. Tests are claimed to be objective, reliable, useful, and cost-efficient ways of 
obtaining information about students' achievements, behavior, attitudes, values, and 
satisfactions. The variety and possibilities for such tests are enormous: they may be criterion- 
referenced (all students pass at a designated level of performance) or norm-referenced; they 
may be used for individual diagnosis or group description; or they may be locally designed or 
produced by major testing companies. 

Standardized tests have been frequently used for comparative purposes so that colleges 
can determine how well their programs and students are doing in regard to norm-referenced 
test results. Standardized tests beyond those measuring entry-level basic skills are relatively 
scarce, especially when a college seeks to understand outcomes from general education or 
higher level intellectual skills. Too often a "quick fix" mentality has promoted the 
inappropriate use of simple, low-cost, paper-and-pencil tests that provide information on a 
narrow range of subject items or basic skills. Such testing frequently "trivializes the 
curriculum" and promotes "teaching to the test" as a strategy for improving student 
performance, masking underlying limitations of the test instrument. 

Northeast Missouri State University is probably the most experienced institution in using 
norm-based standardized tests over the past 15 years. Northeast uses the ACT assessment 
tests, ACT-COMP objective tests, and GRE tests in a very large number of subject matter 
field 4 * In addition, Northeast's faculty have developed t* ^ir own local tests and use a number 
of surveys of the background characteristics, attitudes and experiences of students as they 
progress in their courses of study. 

New Jersey has administered a College Basic Skills Placement Test for over a decade to 
ail entering ^reshmen at public colleges. Results from the New Jersey basic skills test show 
very low levels of proficiency; only 26 percent appeared to be proficient in verbal skills: 28 
percent appeared to be proficient in computation; and only 12 percent appeared to be 
proficient in elementary algebra (Lutkus, 1985, p. 8). New Jersey has also developed a College 
Outcomes Evaluation Program which includes the collection and reporting of information 
about student learning; student development/post-collegiate activities; research, scholarship, 
and creative expression; and community/society impact. Within the student learning category, 
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New Jersey is focusing on five major areas: basic skills; general intellectual skills; modes of 
inquiry; appreciation for the human condition and ethical issues; and application to study in- 
depth or in the major (New Jersey, Advisory Committee Report, 1987). 



For further information on Northeast Missouri, CONTACT: 

President Charles J. McCIain 

or Darrell W. Krueger 

Dean of Instruction 

Northeast Missouri State University 

Administration 203 

Kirksville, MO 63501 

or READ: In Pursuit of Degrees With Integrity: A Value-Added Approach to 
Undergraduate Assessment. Prepared by Northeast Missouri State University 
and published by American Association of State Colleges and Universities, 
1984. Also, The Value-Added Program: Assessing Undergraduate Education, 
prepared by Northeast Missouri State University for the National Governors' 
Association Task Force on College Quality, St. Louis, Missouri, December 
1985. 



For New Jersey, CONTACT: 

Anthony Lutkus, Director of the New Jersey Basic Skills Assessment Program, 
and Edward A. Morante, Director of the College Outcomes Evaluation 
Program, New Jersey Department of Higher Education, 225 West State Street, 
CN542, Trenton, NJ 08625 

or READ: Report to the New Jersey Board of Higher Education from the 
Advisory Committee to the College Outcomes Evaluation Program, October 1987; 
and Lutkus, "Testing and Reporting on Graduates: The New Jersey Basic Skills 
Assessment Program," in Daly, College- School Collaboration: Appraising the 
Major Approaches, New Directions for Teaching and Learning, No. 24, San 
Francisco: Jossey-Bass, 1985. 



Assessment as Competency-Based Education 

This model of assessment identifies particular skills and competencies a college wul 
develop in its students as part of the college experience. Students are then measured in some 
way to determine whether they have acquired the appropriate competencies. A main 
assumption of the competency-based approach is this: if one knows the main components of 
an effective performance in college or outstanding job performance after college, then one can 
systematically teach students to be effective performers. Competency-based education also 
assumes that the specific and general outcomes that are to bo achieved along with the methods 
to achieve those outcomes can be clearly stated and assessed (Grant, et al, 1979). 
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Competency-based education is important because it focuses on achievement of desired goals 
rather than on the time spent in classrooms or the credits accumulated or the educational 
process used to achieve results. 

Alverno College, a small private four-year liberal arts college locate'd in Milwaukee, 
Wisconsin, is a noteworthy example of a competency-based program. Alverno has set forth 
eight areas for student achievement: communication; analysis; valuing in decision-making; 
social interaction; taking responsibility for the environment; involvement in the contemporary 
world; and aesthetic response. Faculty determine the criteria for assessing these competencies 
and use multiple methods (including student performances, community evaluators, simulated 
exercises, extensive interviews) to judge the level of student performance. Alverno's 
commitment to using assessment as a form of instruction and providing continuous feedback to 
the learner sets this college apart from others in its ability to promote student learning. To 
make assessment work for the learner, a strong series of connections must be made: 

[EJxpected outcomes must connect to criteria for performance, to assessment 
practices, to instructional strategies. On a day-to-day basis, these connections 
translate into relatedness between what students learn, how they learn, how they 
wiil be judged, and what their learning means for their future. In a collegiate 
institution, we consider the extent of those connections an important measure of 
the extent to which the environment is organized for learning. We might make 
some of these connections in a single course or program. But the more assessment 
is at the heart of the institution itself, the more its power can serve the learner 
(Loacker, Cromwell and O'Brien, 1986, p. 61). 



For further information on Alverno College, CONTACT: 

President Joel Read 
or Marcia Mentkowski, Director 
Office of Research and Evaluation 
3401 S. 39th Street 
Milwaukee, WI 53215 

or READ: "Abilities that Last a Lifetime: Outcomes of the Alverno 
Experience" by Marcia Mentkowski and Austin Doherty, AAHE Bulletin, 
February 1984, pp. 5-14; and "Assessment in Higher Education: To Serve the 
Learner," by Georgine Loacker, Lucy Cromwell, and Kathleen O'Brien in 
Assessment in American Higher Education: Issues and Contexts. Washington: 
Office of Educational Research and Improvement, 1986, pp. 47-63. 



Assessment as Learning 

Assessing student iearnin 6 and personal growth is the foundation of most colleges and 
universities' undergraduate education, but many do not employ assessment concepts to 
describe and measure the process. In contrast to assessment of competencies and skills just 
described, assessing student learning and personal development tends to focus mox^ on the 
content or knowledge gained as a result of students' collegiate experience and on the values, 
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attitudes, and motivations of the learners. A classic study using this approach to assessment is 
Astin's Four Critical Years (1977). More recently, the National Institute of Education (NIE) 
Study Group on the Conditions of Excellence in Education argued strongly that one condition, 
assessment and feedback, is an essential ingredient in effective learning and a powerful lever 
for involvement (1984). Regular assessment ?ud feedback constitute a critical means of 
communication that enhances student learning and can improve programs, teaching practices, 
and the environment in which teaching and learning take place. In this view, assessment is an 
organic part of the learning process. More important than assessment as a testing strategy 
where students are sorted and graded for professional certification and licensing, the Study 
Group argues that "assessment has even greater potential as a tool for clarifying expectations 
and for increasing student involvement when it is usee to measure improvements in 
performance" (NIE, 1984, p. 53). A core recommendation of the NIE report stresses the need 
for faculty to be trained in usinf "assessment as a teaching tool." So important is the principle 
of assessment-as-learning that the Study Group argued it requires full faculty participation. 

In two recent papers, Patricia Cross has presented a strong case for bringing the 
assessment effort right intc ih^ classroom (1988a, 1988b). Thinking small about assessment as 
an alternative to large -c^le. :titutional o- statewide mandated testing, Cross sees the 
faculty, not only as the "deci^t^ guardians of standards and quality," but also as the 
fulcrum of change in curriculum, teaching strategies, and the quality of student learning. 
"What happens when teachers meet students in the classroom lies at the heart of educational 
quality" (Cross, 1988a, p. 29). Calling for faculty to become classroom researchers, she argues 
that such faculty can experiment and develop more effective teaching strategies based in their 
own disciplines, with the types of students they teach, and focus on matters that are most 
relevant to their teaching goals. Such an effort is likely to generate more substantial 
improvements ia learning if teachers can acquire the proper tools to assess students' learning 
in their classrooms and conduct modest expeirments on the impacts of their teaching. 

Cross points out that large gaps may exist between teaching and learning: 

Teaching and learning are not necessarily two complimentary aspects of the same 
phenomenon. Learning can and does go on without teaching. Unfortunately, too, 
teaching can and does go on without learning. But while learning has many ends, 
teaching has only one: to enable or cause learning. 

I am not suggesting that teachers can close the teaching/learning gap without the 
cooperation of students. Learners share in the responsibility for the effectiveness 
of education. Much as a zipper involves both sides equally in moving toward 
closure, so classroom research seeks to mak/* teaching and learning equal partners 
in narrowing the gap between teaching and learning. 

By the same imagery, however, the zipper becomes stuck when teachers believe 
♦hat their sole obligation is to present material in a clear and logical manner. 
Unless teachers can make connections between their teaching and what students 
already know, learning is not likely to follow. The purpose of classroom research is 
to provide continuous feedb r ck on what students know and learn, so that teachers 
and students can relate to o^e another by making all those little connections that 
move teaching and learning closer together (Cross, 1988b, pp. 3-4). 
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For further information, READ: 

K. Patricia Cross and Thomas Angelo, Classroom Assessment Techniques: A 
Handbook for Faculty. Ann Arbor: University of Michigan, NCRIPTAL, 1988; 
K. Patricia Cross, "Feedback in th<* Classroom: Making Assessment Matter," 
paper presented at AAHE Assessment Forum, Third National Conference on 
Assessment in Higher Education, Chicago, Illinois, June 8-11, 1988a; and K. 
Patricia Cross, "In Search of Zippers," AAHE Bulletin, Vol. 40, No. 10, June 
1988b, pp. 3-7. 



Assessment as Value-Added 

The value-added approach goes beyond the idea of testing at the end of the program or 
completion of a major or the first two years of general education. This approach requires 
assessment of students' skills and knowledge and attitudes at time of entry and then at 
subsequent points during the students' college careers. By having in effect pre- and post- 
measures, the model controls for the entering abilities and knowledge of the students so that 
the contribution of the college's program can be assessed in the change students make as they 
move to graduation. 

The University of Tennessee at Knoxville employs the ACT-COMP test with freshmen 
and seniors so that college impact can be evaluated. In addition, seniors are assessed by 
depauments regarding knowledge of their major field, and opinion surveys supplement test 
scores. All of this value-added data is designed to satisfy the state of Tennessee's requirement 
for outcomes information that is tied to a statewide performance funding program. Colleges 
must demonstrate their impact on students through outcomes assessment which has funding 
implications based on the results. (See Fisher's presentation (p. 52) and Banta's presentation 
(p. 58) for the most recent picture of Tennessee's approach as well as the resources listed 
below.) 



For further information on the University of Tennessee at Knoxville and the 
Tennessee Performance Funding Approach, CONTACT: 

Trudy S. Banta, Director 
Assessment Resource Center 
University of Tennessee, Knoxville 
2046 Terrace Avenue 
Knoxville, Tennessee 37996 

and 

Homer S. Fisher 

Executive Vice Chancellor for Business, Planning and Finance 
University of Tennessee, Knoxville 
405 E. Andy Holt Tower 
Knoxville, Tennessee 37996 

or READ: Banta and Fisher, "Putting a Premium on Results," Educational 
Record, Spring-Summer 1986, pp. 54-58; Banta, "Use of Outcomes 
Information at the University of Tennessee, Knoxville," in Peter T. Ewell (ed.), 
Assessing Educational Outcomes. San Francisco: Jossey-Bass, Inc., New 
Directions for Institutional Research, No. 47, 1985, pp. 19-33; Banta, (ed.) 
Performance Funding in Higher Education. Boulder, CO: National Center for 
Higher Education Management Systems, 1986. 



Assessment as Program Evaluation 

Using assessment as a program evaluation strategy is a common way for colleges and 
universities to determine their effectiveness and to collect information relevant to the decision- 
making process. In this approach, the program is the unit of analysis and not the individual 
student who progresses through a course of study. A program may be defined as a 
departmental major, or the general education curriculum, or the entire degree earned. Usually 
these assessments are done for internal management reasons and are formative in nature 
although they may produce data useful for external accountability. Program evaluations are 
built around the goals to be as ^ssed and frequently use multiple measures to provide feedback 
for program improvements. With the arrival of strategic planning in higher education 
administrative thinking, some colleges are now conducting program evaluations which include 
a cycle of program planning and development, implementation, evaluation, and budgetary 
decision making. Such a cycle, with an assessment component, allows actual data on program 
goals, operations, and effects, to be included in overall decisions about the institution's future 
directions. 

Program evaluation studies have been conducted for over fifty years in higher education. 
Pace and his colleagues at the University of Minnesota initiated probably the first elaborate 
program evaluation studies during the late 30s (Pace, 1941). Pace was inquiring into the 
effectiveness of the general education programs. 
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A recent example of assessment as program evaluation can be found at Empire State 
College, an innovative college serving adult learners in more than 40 locations across New 
York. Called PERC (Program Effectiveness and Related Costs), this strategy has four unique 
features: it focuses on assessing educational outcomes first and then relates costs to th^se 
outcomes; it relies upon a multiple perspectives strategy; it calls for conducting studies within a 
longitudinal design; and it emphasizes the practical utility of those results for academic 
decision makers. Assessment efforts are focused on providing answers to this question: What 
kinds of students working with what kinds of faculty in what kinds of learning programs change 
in what ways at what costs? Embedded in this master question are the five concepts of 
PERC— outcomes, costs, students, faculty, and programs. Reports have been prepared which 
identify ways to improve the teaching/learning process, increase the retention of students, 
improve the effectiveness of program operations and delivery, and eventually affect the 
strategic planning process at the college. 



For further information, CONTACT: 

Timothy Lehmann 

Assistant Vice President 

Office of Research and Evaluation 

Empire State College 

One Union Avenue 

Saratoga Springs, NY 12866 

or READ: Ernest G. Palola and others* The PERC Handbook. Saratoga 
Springs, NY: Empire State College, 1977; Timothy Lehmann, "Evaluating Adult 
Learning and Program Costs," in Arthur Chickering and Associates, The 
Modern American College, San Francisco: Jossey Bass, Inc., 1981, pp. 748-772; 
and "SUNY Empire State College," in Jacqueline Paskow (ed.), Assessment 
Programs and Projects: A Directory. The American Association for Higher 
Education Assessment Forum, Washington, D.C., 1988. 



Assessment by External Examiners 

This approach to assessment draws upon external examiners, usually faculty from other 
colleges, to examine graduating seniors. Relying upon expert peer judgment, this approach to 
assessment overcomes the usual practice in which the faculty that do the teaching are the same 
ones evaluating the students' learning. In addition to the major advantage of separating 
instruction from evaluation, this approach examines learning beyond what is acquired in a 
course, necessitates faculty participation in the process, considers the distinctiveness of a 
particular program or department, provides for comprehensive and often multiple forms of 
feedback, and allows for assessment of higher-order cognitive skills (Fong, 1987). This is an 
mternal form of professional review, captured well by John Harris's aphorism, "Somebody 
besides the cook should taste the pudding." 

')fr 
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External examiners can help a department or program of a college understand the 
knowledge acquired and skills gained by its senior majors and can pinpoint weaknesses in the 
department's operations. This strategy provides an important capstone experience for 
students, giving them sound feedback on their performance?. To implement such a strategy, 
the department or program must be clear about its educational objectives so that its own 
faculty, students, and the external examiners are able to understand what objectives are to be 
achieved and what learning experiences are io be examined. The arts programs in many 
colleges today draw upon this strategy, as illustrated in the music major's senior recital and the 
art major's culminating portfolio reviewed by practicing artists. This assessment strategy goes 
beyond the usual credit-hour accumulation for graduation by requiring seniors to demonstrate 
their knowledge and abilities to outsiders whe make judgments about their performance. 
Swarthmore College and Alverno College are well known examples of American colleges 
relying upon this approach to assessment. In 1986, the Association of American Colleges 
launched a major project involving 18 colleges and 11 disciplines to field-test the applicability 
of external examiners in a variety of educational situations. 



For further information, CONTACT: 

Carol Schneider 
Vice President 

Association of American Colleges 
1818 R. Street, NW 
Washington, D. C. 20009 

or 

Marcia Mentkowski 

Director, Office of Research and Evaluation 
Alverno College 
3401 S. 39th Street 
Milwaukee, Wisconsin 53215 

or READ: Bobby Fong, "The External Examiner Approach to Assessment," 
paper presented at the Second National Conference on Assessment in Higher 
Education, AAHE Assessment Forum, Denver, Colorado, June 14-17, 1987. 
Oil Swarthmore College, see Ohmer Milton and John Edgerly, The Testing and 
Grading of Students. New Rochelle, NY: Change Magazine, 1976. 



Assessment as Self- Assessment 

Another perspective on assessment involves the learner engaging in a series of self- 
assessment activities and exercises with the ultimate £oal of becomin$ an independent, self- 
directing person. One of the goals of undergraduate education at most colleges and universities 
is for the students, by the time they reach their senior year, to have gained enough knowledge 
and awareness of the learning process itself so that they can continue beyond college to be 
independent learners, employing critical thinking and evaluative skills learned in college. As 
college programs begin to recognize individual differences and provide educational strategies 
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to sustain particular students' learning interests, there have been a number of assessment 
techniques developed to foster student independence in learning- Malcolm Knowles, for 
example, in a recent book on Using Learning Contracts (1986) presents a number of 
competency diagnostic guides, planning guides, checklists for learning, and study plans. Other 
advocates of learners practicing self-assessment activities have produced learning style 
inventories, learning project efforts, life cycle exercises, and other career and skill building 
inventories (Kolb, 1984; Tough, 1971, 1978; Knox, 1987). 

Although many colleges proclaim that the goal of creating self-directed learners is an 
important one, they do not rely exclusively on self-assessment strategies like those mentioned 
above. Rather, they combine some techniques like the ones identified above with other 
assessment approaches previously discussed. One of the most critical skills that can come out 
of the self-assessment effort is self-evaluation. Every learner needs to understand the 
evaluation process and recognize the personal biases inherent in any evaluation effort; yet 
most learners need to move from a more distaut, objective evaluation strategy contained in the 
other assessment perspectives to a thoughtful, mature, informed and critical self-evaluation 
process that makes learning significant to the individual. 



For further information, READ: 

Malcolm S. Knowles, Using Learning Contracts. San Francisco: Jossey-Bass, 
Inc., 1986; David A. Kolb, Experiential Learning. Englewood Cliffs: Prentice 
Hall, 1984; Alan Knox, Helping Adults Learn. San Francisco: Jossey-Bass, 
Inc., 1987; Allen Tough, The Adult's Learning Projects. Toronto: Ontario 
Institute for Studies in Education, 1971; and Tough, "Major Learning Efforts: 
Recent Research and Future Directions." Paper presented at the American 
Association for Higher Education National Conference, 1978. 



Assessment as Accreditation 

Regional accrediting agencies have traditionally reviewed colleges and universities 
according to broadly defined criteria that ha^e reflected resource and reputational models of 
excellence. The resource model of excellence, for example, posits that student learning and 
development is most likely to occur when the right resources are brought together (students, 
faculty, physical facilities, and financing). The basic assumption here is that the more 
resources there are the better the learning is. The focus is on such measures of quality as low 
student/faculty ratios; high student test scores; high faculty publication rates; low average class 
size; large numbers of library books; and high endowments. 

In support of reforming undergraduate education, critics argue that the old models of 
accreditation are no longer sufficient and should be replaced by a new focus in accreditation 
self-studies and evaluation visits. Calling for a new focus on performance, results, outcomes, 
and achievements, the reform movement asks regional accrediting associations to assess 
colleges and universities in determining the "kinds of instructional outcomes that the institution 
and its programs seek to produce; documenting the processes by which the institution ensures 



that these goals are in fact being attained, together with the appropriate evidence of 
attainment; and providing evidence of how the results of these processes are actually used by 
the institutions to make improvements* (Ewell, 1987, pp. 6-7). 

The focus on outcomes has now been picked up by most regional accrediting 
associations- For example, the Middle States Association of Colleges and Schools, in 1982, 
published Characteristics of Excellence in Higher Education, which contains a separate section 
on outcomes. Middle States says: "The deciding factor in assessing the effectiveness of any 
institution is evidence of the extent to which it achieves the goals it has set for itself" (1982, p. 
10). Using language carefully chosen to avoid reliance on any one method of measurement or 
preferred assessment strategy, Middle States suggests a multiplicity of approaches to assess 
student progress and achievement and calls for a pl*n to measure outcomes that reflects an 
institution's continuing concern with evaluation and with using results to improve the 
teaching/learning process. 

Professional accreditation agencies assess particular programs from the focused 
perspective of a given occupation or profession. Outcomes assessment here is usually a generic 
part of the certification-to-practice process and program graduates are asked to demonstrate 
their professional competence. Outcomes assessment is recognized as important and legitimate 
for several reasons. First, it is acceptable because professional accreditation signifies 
accountability to a professional community and indirectly to the larger public. Second, 
outcomes of professional education are more limited in nature and somewhat easier to define 
and assess. Third, professional training and assessment of student work is closely tied to 
professional certification. 

To illustrate the outcomes focus in a professional area, the American Assembly of 
Collegiate Schools of Business has been conducting an outcomes measurement project over the 
past decade and recently completed Phase m of its work. Phase I identified and classified the 
knowledge, skills, abilities, aptitudes, personal characteristics and values that every business 
school graduate should possess. Phases II and EI represented major efforts to develop 
instruments and field-test those instruments to determine their effectiveness in measuring the 
desired attributes of business school graduates. The major knowledge areas for which tests 
were created included: accounting; business environment and strategy; finance; human 
resources ^nd organization theory; marketing; management information systems; and 
quantitative analysis. Nine dimensions of skills and personal characteristics were also 
identified and a set of behavioral simulations were designed to measure the characteristics. 
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For further information, CONTACT: 

Milton R. Blood 
Director of Accreditation 

American Assembly of Collegiate Schools of Business 
605 Old Dallas Road, Suite 220 
St. Louis, Missouri 63141 

or READ: American Assembly of Collegiate Schools of Business, Outcomes 
Measurement Project, Phase m Report, May 1987; Characteristics of Excellence 
in Higher Education, (1982), and Handbook for Institutional Self-Study (1984), 
Commission on Higher Education of the Middle States Association of Colleges 
and Schools, Philadelphia, PA- Peter T. Ewell, "Assessment, Accountability 
and Improvement: Managing the Contradiction," paper prepared for AAHE, 
National Center for Higher Education Management Systems, Boulder, CO, May 
1987. 
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m. ASSESSING PERFORMANCE IN HIGHER EDUCATION: 

The Norstar Conference 



The conference from which this Conference Proceeding emerged was designed to 
convene New York policymakers from government, business, and the higher education 
community to hear from the experts the national, state, and institutional perspectives on 
assessment. Drawing upon the Norstar Bank's endowment for an annual conference on 
excellence in education, the authors of this report developed the conference on "Assessing 
Performance in Higher Education," which was held June 23, 1988. The first two sections of the 
report have provided the context for the current trend towards reform in undergraduate 
education, and have presented four models of excellence and eight approaches to assessment 
to guide those interested in the issue of improving the quality of higher education. What 
follows are the presentations of the experts. It shorld be noted that the presentations are 
edited transcriptions of the speeches delivered at the conference and should be read, not as 
post-conference papers for publication, but as presentations to this audience. The speeches in 
this section are intended to frame the discussion and contribute to the policy process as New 
York State considers the issues of accountability and assessment. 
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Opening Remarks 



James W. Hall 
President 
Empire State College 
Saratoga Springs, New York 



I want to welcome you on behalf of the Rockefeller Institute *nd Empire State College, 
cosponsors for this conference. We are also supported by a Norstar Corporation endowment 
and we are much indebted to them. 

This Institute is a very special place. The concept has been to provide a place where 
persons from government, from the private sector, from the voluntary sector, and from 
different areas of the state can come together to discuss important, often controversial, issues. 
A place where your comments are in this room, where you can talk candidly and frankly about 
the important issues of our time, and a place where hopefully we can reach new 
understandings. 

Today's topic is assessing performance in higher education. This topic certainly is at the 
center of the agenda in New York State. Ted Fiske, from The New York Times, who is here 
with us today, has pointed to the expected increase of federal interest in education at all 
levels. Given the scarcity of funds at the federal level and also in the state of New York, it's 
clear that public demands for a better understanding of what happens in higher education, and 
for increased accountability from its institutions, is needed as requests for additional funds 
occur. So it is an important topic that you are gathered to talk about. 

The context *or this topic is a rich and important one. Probably no nation in the world 
has made such a commitment to higher learning as this country has from its very beginning. 
Harvard College, ifter all, really wasn't sufficient. It sufficed for about 60 years. But soon we 
needed a number of colonial colleges; during the 19th century we needed to build colleges by 
every description, following the moving western frontier. Every town wanted to have a college 
of its own so there would be an opportunity for people to learn and move beyond a grammar 
school education. We are all familiar with the land grant movement, which broadened the 
curriculum and brought more people into the university, and the development of the great 
research universities around the end of the last century, and more recently the coming of the 
community college, a very important broadening movement in higher learning, sparked by the 
GI bill and upward mobility. Certainly in New York State the development of a strong 
independent sector and a strong public sector are indicative of the kind of support and 
commitment that people have made to higher learning. 

It has been a dual commitment, with two seemingly irreconcilable <iims. On the one 
hand, there has been a commitment to quality, to strong excellence in our institutions. On the 
other hand, there has been commitment to access and opportunity. Those two have played 
against one another throughout the entire history of higher education in America. Today, in 
New York State, about 70 percent of all high school graduates go on to some form of college 
and more of them go later on as adults. 
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In recent years, however, we have seen many reports that question the quality of what 
those of us in colleges and universities do. And the question of whether real access is working 
for many citizens remains. So questions about whether both quality and access are achievable 
are now widely asked. Certainly, many are asking for improved evidence Lhat colleges are 
doing the job, that the resources that are given to our institutions of higher learning are used 
responsibly. They want assurance that we are candid and accountable in reporting individual 
student performance, in looking at the performance of different academic programs, and even 
at the effectiveness of entire institutions. 

There are a small number of institutions in this country where a number of talented 
students go. But most students, and many of them are among are most talented, go to 
institutions that do not carry the names of those colonial colleges. These are the institutions 
where the workforce of this state is trained and educated. These are the places where the 
people who will power our collective future have received their educations. I am talking about 
the Fredonias, the Hartwicks, the LaGuardias, the Hudson Valleys, the Piatt sburghs, and as 
you get older, Empire State College. The performance of these public and independent 
colleges is really csntral to the success of this state and its many other institutions. 

But how do we know-how these colleges perform? How do we measure how they 
perform? You are senior policy administrators and analysts who come from many different 
institutions — higher learning, government and other agencies in this state. We are asking you 
today to engage and deliberate these questions. After all, it is your participation in this 
beautiful building, which makes this Institute a wonderful idea. At stake ultimately, I think, is 
the grand American experiment in quality and access. And the whole world is watching how 
we Americans do. So let's get underway. 
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Accountability for New York State Agencies 



Thea Hoeth 

Special Assistant to the Governor for Management and Productivity 

Albany, New York 



As you share with each other your knowledge and insights about a very timely and 
important topic, I'd like to share with you some of my experiences. Early last March when I 
was wearing my previous hat, I joined with some of you to discuss thw evaluation of state 
programs and, in particular, how to design evaluation systems for higher learning- I commend 
all of you for coming today to bring together your knowledge and expertise in this area. I 
commend the Rockefeller Institute for causing this gathering. 

I would like to take just a bit of your time to talk about some ways that New York State 
is approaching the improvement and evaluation of its programs. If it's the "Age of Aquarius" 
in Washington and, of course, the "Decade of the Child" in New York State, then it's the 
"Age of Accountability" in Albany. In his State of the State message this year, Governor 
Cuomo said: "Heightened accountability is much more than additional auditors and new 
attention to internal controls. Throughout the executive branch, I will insist on pervasive, 
profound, and tangible efforts to improve the efficiency and effectiveness of State agencies." 

The Office of Management and Productivity has worked with agencies since 1983 to help 
them establish their own internal systems for monitoring operations and for creating an 
environment in which doing more with less is the norm. We work with agencies in the area of 
internal information systems, which is a critical element in monitoring and controlling agency 
operations. For example — we are working with the housing agencies to establish accurate data 
about spending for housing production and renovation; we are working with the Department of 
State to automate its corporation and uniform commercial code records; and we are helping 
the Banking Department computerize its audit functions. These perhaps unglamorous steps are 
nonetheless fundamental. And they begin paying off immediately in smoother agency 
operations and improved service to the public which, after all, is the point of all this. Those 
agencies involved will need to build on these initiatives and evaluate what changes or 
additional steps they need to take in the future. 

There have been many other significant process improvements that we have been 
working with agencies to design. As far as evaluation, I'd like to tell you about the launching 
of a first- of-its-kind effort in New York State with 15 of the state's largest agencies. We 
include in that 15 the State Education Department (SED), the State University of New York 
(SUNY) and the City University of New York (CUNY). The governor is asking these agencies 
to provide accountable information to display the worth of their programs. We are asking 
agencies *o identify their most important programs, and then to analyze those programs and to 
measure their effectiveness to determine whether our investments are worthwhile. 

The Office of Management and Productivity, the governor's program staff, and the 
Division of the Budget will study the agencies' replies to evaluate successes or failures. In the 
process, we hope to get a better picture of the state's overall operation. The vehicle by which 
this process is occurring begins with agency "report cards" or operations reports, which are 
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already in and are under scrutiny. The results of this analysis will ultimately be reported to the 
public and that is an important element in all this. 

The inclusion of SED, SUNY and CUNY in the first round of 15 agencies is significant. 
Judged by financial commitment, education is a top state priority. After all, state support for 
elementary and secondary education has increased by over 65 percent in the last six years, 
and SUNY and CUNY rank first and third, respectively, as the largest higher education 
systems in the country. 

What we have found over the years, however, despite those large investments, is that we 
still have problems with our educational systems and in determining what the state should be 
getting for its investment. You, involved in education, know better than anyone else the most 
pressing of the problems: How difficult it is to curtail the dropout rate; to give disadvantaged 
students additional support; to help make their school years a success; and to provide students 
with necessary skills to become productive participants in a dramatically changing workforce. 

This last problem— providing students with skills to perform well in the workplace- 
presents employers of the state with as much of a dilemma as it does the students. Last week, 
in the same room, I participated in another conference which brought together public and 
private human resource specialists to discuss the public workforce of the 21st century and how 
we should begin preparing for that. Private employers do not hesitate to complain about the 
diminished skills of young people entering the workforce. The public sector, I assure you, is no 
less alarmed. How do we attract and retain quality teachers who get personal satisfaction 
from their jobs when we offer them little financial incentive? The governor has suggested many 
ideas — financial bonuses, more educational research opportunities, educational resource 
centers — but we have yet to come up with a workable system that sufficiently rewards those 
dedicated professionals. 

Let me share with you some highlights from SUNY's and CUNY's operations reports, if 
I may, concerning how they are addressing the special problems of disadvantaged students 
who need supportive services and special courses to help them succeed. SUNY has made a 
special effort to recruit and assist minority students. In its operations report, SUNY notes that 
the number of minority undergraduate students rose from 7.7 percent of total enrollment in 
1976 to 11.5 percent in 1987. The report disclosed SUNY's disappointment, however, with low 
numbers of minorities receiving graduate degrees. Although this follows the national trend, the 
governor and legislature responded to this need and increased the Graduate Fellowship 
Program, which was initiated last year. The university will also focus attention on attracting 
more minorities and women for administrative and faculty positions, and will work with the 
Governor's Office of Employee Relations and the United University Professions (UUP) to 
devise recruitment and retention strategies. The most successful program in recruiting minority 
students involves a "partnership" between the high schools and the colleges. According to 
SUNY's report, for example, SUNY students and professors are now involved in about 300 
partnerships with high schools, providing such services as counseling and exchanging ideas on 
curriculum development. 

CUNY's efforts have also been ambitious in this area. In its operations report, CUNY 
shows that the proportion of minorities to total enrollment has more than doubled since 1970, 
with minorities comprising 62 percent of all students in the university in 1986. I find it 
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interesting that the profile of CUNY's "freshmen" presents the following picture of the 
students whom the university seeks to serve: 37 percent of 1986 first-time freshmen were older 
than 19 years; 17 percent were 25 years or older; 58 percent were women; and even upon 
college entry, about half of all 1980 freshmen held jobs, with 13 percent working full time. 
According to the CUNY report, many of these entering students are academically 
underprepared, with only 22 percent passing three required skills assessment tests and 27 
percent failing all three initially. To compensate, many freshmen take remedial courses. In 
1980, for example, half of the entering students took at least three such courses in the first 
year. 

In fact, CUNY's most successful effort with these underprepared students has been 
through the PreFreshmen Summer Program, started in 1985. Conducted in the summer for 
freshmen enrolling in the fall, this program provides basic skills in reading, writing and 
mathematics as well as an orientation to college life. The idea, of course, is to beef up 
students' skills and ease their transition to college. Although still in its early stages of 
evaluation, the PreFreshmen Program apparently is generating results. According to CUNY's 
operations report, participation in the program has climbed over the past three years from 
about 500 to 2,700 students. More than 90 percent of the total 4,600 participants have 
completed the program and subsequently enrolled as freshmen. These students are displaying 
marked improvement in basic studies, and setting a precedent — that many of them desperately 
n<*ed — for continued academic success. 

Of course, lists of difficulties in the education area continue. And some may wish to see 
these not as difficulties but as opportunities and challenges. I hope I have conveyed to you the 
premium this administration places on the need to assess all that we do and its special interest 
in the education we provide to our children and to our adults. I again commend you for your 
courage in gathering together today to address these challenges and for the work you do 
everyday in this area. 
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Academic Assessment: 
The Right Thing for the Right Reason 



Joseph C. Burke 
Provost, State University of New York 
Albany, New York 



Thomas A. Becket, in T. S. Eliot's "Murder in the Cathedral," warned that "the last act 
is the greatest treason — to do the right things for the wrong reasons," Becket spoke of church- 
state relations in medieval England, but his message has meaning for college-state relations in 
modern America. The comprehensive assessment of undergraduate education is the right thing 
for public colleges and universities. But both the states and their state universities should take 
care that assessment is done in the right way for the right reasons. 

The assessment movement will commit the greatest treason to educational reform, if it 
trivializes the complex and creative outcomes of higher education to fit the inherent limitations 
of standardized tests. It will dash the highest hopes of its best supporters, if it becomes so 
mired in methods of measurement that it fails to develop a new and broader vision of 
institutional quality that combines excellence with access, It will not meet the critical needs of 
a diverse and democratic society if it homogenizes higher education to suit a monolithic and 
elitist model of quality that fits only a favcred few schools and students. If the assessment 
movement is to do the right things for the right reasons, it must reform, and not just measure, 
undergraduate education. 

Becket's warning should remind us that the reasons for assessment are critical. Why we 
assess determines inevitably what to assess and how to assess it. We must answer each of 
these questions in order, or court misunderstanding or misuse. 

The reasons tor asse**ment are as varied as the motives of its supporters. However, two 
concerns, one external and one internal, drive assessment. One concern stems from external 
accountability to the state or society for the funds supporting higher education. The other 
reflects the internal responsibility of the academy to enhance student learning and to improve 
institutional effectiveness, The coupling of these two concerns constitutes a fragile 
combination — an essential tandem with inevitable tensions. The constant and continuing 
commitment of both the capital ^:_J the campus to assessment requires a delicate balance of 
both concerns. 

External compulsion from the capital can only produce passive resistance on the 
campus. Passive resistance on campus can only produce external compulsion from the capital. 
Yet campus commitment, not external compliance, is critical to assessment. Campus creativity 
comes from internal commitment, not from external commands. Too much external pressure 
can produce campus paralysis. Too little pressure can produce campus complacency. The right 
combination for both capital and campus balance: care with concern, patience with 
persistence. 

Both capital and campus must recognize that in assessment, external accountability and 
internal responsibility arc inseparable. Only an assessment system that demonstrably enhances 
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student development and improves institutional performance can satisfy both external 
accountability and internal responsibility. 

Though there is much debate about the directions and details of assessment, experienced 
practitioners agree that assessment, if it is done well, is expensive and exhausting — expensive 
to the state and society and exhausting to faculty and staff. Case studies suggest that 
assessment is like an iceberg, in the sense that most of the work and much of the value lies 
hidden beneath the surface. Neither the capital nor the campus can afford to measure 
everything that moves and test everything that stands still. Assessment must be comprehensive 
but its results must also be comprehensible and cost-effective. Speed is not a measure of 
success in assessment, for it usually produces superficial results. It purchases little of value at 
the cost of undermining the credibility of assessment on campus and in the capital. 

The general public and state officials quite naturally wonder why assessment should be 
so costly and time-consuming. We should remind them that the questions we ask determine 
the answers we get. If we ask small questions, we can expect short, simple and swift answers. 
If we ask complex questions, we must expect complicated answers. Questions concerning the 
learning of thousands of very different students and the institutional effectiveness of a vast 
variety of colleges and universities on the multiple outcomes of undergraduate education are 
complicated questions. The answers involve assessment of institutional missions in terms of 
campus performance in relation to the needs of their particular students and that segment of 
society they are supposed to serve. 

Assessment must begin with an appraisal of the appropriateness of institutional mission 
and the goals of the curriculum — skills courses, liberal learning, and specialized education — 
and of the range of the academic and student support services in relation to the needs of the 
students enrolled. It can end only with a comprehensive evaluation of the extent to which these 
ends are achieved. The only thing simple about assessment is the answer to the question of 
why it is so time consuming and costly? The simple answer is that the questions of assessing 
student development and institutional effectiveness are incredibly complex. 

But assessment must grapple with a more fundamental question. The true test of the 
assessment movement will be whether it can develop and establish a new definition of 
institutional excellence and quality in undergraduate education — a definition based on 
inclusion rather than exclusion, on performance rather than prestige. The traditional definition 
of institutional quality prevalent on campus and in the country is monolithic and elitist. 
Assessment subverts this vision of excellence. The question is whether the supporters of 
assessment will have the courage to replace it with a broader view of quality that suits the 
higher educational needs of our diverse and democratic society. 

The real revolution in the assessment movement is the threat that excellence will no 
longer be considered the exclusive preserve of the rich and famous institutions. Indeed, some 
of those institutions may be assessed as less than excellent when performance rather than 
prestige is the criterion for success. Surely, it is significant that two of the most prominent 
institutions in the assessment movement are Alverno College and Northeast Missouri State 
and not Harvard and Berkeley. 
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The traditional view of quality in higher education is an amalgam of what Alexander 
Astin labeled the reputational, resource, and selectivity models. According to this view, the 
quality of colleges or universities depends on their prestige and possessions rather than their 
performance — on the quality of the students admitted rather than on what they learn while on 
campus. Conventional wisdom assumes that the "best" colleges and universities have the most 
money, the most faculty with the most publications, the most library books, the most 
programs, and the most students with the highest SAT scores. 

This model of quality presumes that the greater the resources or the "inputs" the better 
the performance or the products of a college or university. It says nothing about student 
development or institutional impact on students. It confuses quantity with quality and 
conceives of the operation of a campus, like that of a computer, as mostly a matter of good 
in-good out. 

Recently, states, society and even some scholars have begun to question the 
appropriateness of this narrow notion of quality. Scholars suspect the validity of assessments 
based on assumptions and assertions rather than on proof and evidence. Howard Bowen, one 
of the leading authorities on higher education, a few years ago studied the correlation between 
funding per stud^rt and student learning in leading public and private colleges and universities 
across the country. He found little relationship between the quantity of campus funding and 
the quality of undergraduate learning. The only conclusion he reached was that institutions of 
higher education always spend all of the money they receive or raise. 

The resource model of quality insults faculty and staff, for it assumes that the success or 
failure of a college or university depends mainly on what they receive in students and support 
rather than on how well they perform. The selectivity model also shortchanges students, for it 
suggests that their success or failure depends mostly on v/hat they know before they enroll. The 
resource model also conflicts with the public's disbelief that quantity funding automatically 
guarantees quality performance. The public has seen in recent years report after report 
questioning the quality of undergraduate education despite sizable increases in funding. 

Finally, this monolithic and elitist model of excellence seems unrealistic and 
undemocratic, for it sets a standard that makes the pursuit of excellence an impossible dream 
for most colleges and universities and for most students. The reputational and resource model 
of excellence segregates schools and students into two classes — the favored few and the 
mediocre many. 

The feeling is growing throughout the country, especially in state capitals, that this 
traditional view of quality is fatally flawed. Increasingly, public officials are arguing in the 
name of accountability that the quality of colleges and universities should be judge _ on their 
outcomes rather than on their incomes. And more educational experts are asserting that the 
true test of institutional quality of a school should be its contribution, or the value added, to 
the intellectual and personal development of its students from admission to graduation. Both 
groups agree that the quality of colleges and universities should be judged not by their capacity 
to recruit the brightest students or to raise the most resources but by their ability to assist their 
students to become the best they are capable of being. This value-added approach can supply 
the missing link between equality and quality, access and excellence. 
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The dilemma of American democracy has always been how to reconcile equality and 
quality? How to shape a diverse and democratic society that is both equal and excellent? How 
to ensure equality of opportunity for all Americans and yet spur superior performance in a 
pluralistic society where individuals differ widely in abilities and circumstances? At times, our 
search for both quality and equality seems an impossible dream. We waiver between these two 
ideals, shifting first to one and then to the other. Some fear our resolve for equality will result 
in the "mediocrity of the many," while others feel our push for quality will produce the "elitism 
of the few," Despite their differences, these divergent views share the common but mistaken 
belief that we as a people can seek one of these goals only at the expense of slighting the 
other. 

John Gardner, the founder of Common Cause, in his book, Excellence, wrote eloquently 
on this subject of equality and quality. His book should be required reading in every public 
college or university in the country and should become an important document in the debate 
on assessment. Gardner reminds us that our crusade for both equality and excellence need not 
be a search for an impossible dream. To Gardner, equality and excellence were not the source 
of an unsolvable dilemma but of a dynamic interaction where one goal enriched the other. 
Gardner saw education as the indispensable link between equality and quality. 

In an earlier age, an elementary and then a high school education for most Americans 
was adequate to ensure that our society was both equal and excellent. But the growing 
demands of learning and life in a society based increasingly on knowledge and information 
require that a growing proportion of Americans benefit from higher education. Despite the 
bickeiings of Bill Bennett, higher education has a public benefit for society as well as a private 
benefit for students. 

Economic and demogiaphic trends will demand that more citizens must receive more 
and better education at the associate and bachelor's levels. And minorities — especially blacks 
and Hispanics, who have benefited the least from education at all levels — will comprise a 
much larger proportion of the work force and require more higher education. Student bodies 
on most campuses will become ever more diverse — more part-time, more minority, more 
mature, more handicapped. The residential college in a rural setting filled with recent high 
school graduates with good educational preparation and clear career goals has become the 
exception rather than the rule. Yet the conventional view of institutional excellence in higher 
education lingers on though the conditions that fostered it have long since gone. 

For the State University of New York, the goals of quality and equality, access and 
excellence, represent a challenge — not a choice. SUNY believes that a public university that is 
elitist and exclusive is a contradiction in terms. No public university should accept the 
aristocratic assumption ^that the only entree to excellence is through exclusivity — that 
excellence is achievable pnly by limiting access. Neither should a public university accept the 
contrary thesis that educational inferiority is the inevitable consequence of access — as though 
most students and most institutions can never aspire to excellence, for quality in higher 
education is by nature reserved for the favored few schools and students. 

A value-added approach to assessment combines access with excellence by insisting that 
each student and each institution strive to become the best they can be. This approach seems 
appropriate for the State University of New York, which was founded 40 years ago with the 
original motto: "Let each become all he (or she) is capable of being. n 
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Assessment and value-added seem the right things for SUNY. But before prescribing a 
monolithic model of assessment on the biggest and mowt diverse university in the world, we 
should remember the quip of H. L. Mencken — that for every complex problem there is a 
simple solution that's wrong. No single assessment method can fit the SUNY system. Our 
university enrolls 333,000 undergraduate students, nearly 40 percent of whom are part-time; 38 
percent of the new students each year are transfers. The 64 campuses of SUNY are as diverse 
as the students they enroll. They include community colleges, technical colleges, specialized 
colleges, statutory colleges, comprehensive colleges, and university centers with a wide range 
of educational missions, academic programs, and admission standards. 

The diverse needs of New York State demand this diversity of campuses in its state 
university. Any assessment effort must enhance, not diminish, the distinct mission of each of 
these campus categories. No monolithic model of assessment mandated frjm Albany can fit 
this range of institutions with their diverse missions. 

Despite the differences among SUNY campuses, all share a common commitment to 
educational excellence and institutional assessment. SUNY was built on the belief that each of 
our campuses must aspire to educational excellence in terms of its performance of its 
designated mission. The test of excellence in undergraduate education for ea.Ii of our 
campuses is its contribution to the intellectual, social, and personal growth of its students from 
admission to graduation. 

We are trying to take advantage of the system aui of its categories of campuses by 
collecting comparable data on student perceptions of the quality of their undergraduate 
experience, by fostering experiments with different approaches to assessment, and by sharing 
throughout the system the experiences of our campuses in evaluating institutional performance. 
The size and diversity of SUNY allows the collection and analysis of a wealth of comparative 
data from all types of students and campuses. 

The central administration in cooperation with the campuses has instituted on a three- 
year cycle the survey of the American College Testing Service that assesses student 
satisfaction with academic programs and campus services. The survey results have been 
especially useful both on campus and in the central administration in comparing the student 
ratings of services among campuses in the same category o f SUNY institutions. The results 
have been used by the campuses and the central administration in preparing program 
proposals and budget initiatives. 

The categories of institutions within SUNY alsj allow special assessment efforts through 
cooperative and consortia arrangements. Though a number of these efforts are under way, I 
will discuss briefly only a few assessment program^ at the arts and science colleges. Of course 
one of those colleges, Empire State, our host institution today, has long been involved in a 
unique form of assessment. Indeed, it offers a perfect conjunction of individual student 
development and institutional performance, since that college tailors its academic programs to 
suit the individual needs of each student. 

I could say much more about this fascinating process of assessment but simple justice 
dictates this be done by the founding president of this unique college, who is our moderator. 
Instead, I will concentrate on two programs, one at Plattsburgh — where I spent twelve years as 



ERLC 



~ 33 ~ 40 



president — and the other at Fredonia. Both of these projects are currently supported by grants 
from the Fund for the Improvement of Post-Secondary Education. 

The Plattsburgh experience, and other case studies, suggests that effective assessment 
programs are not built on a single action but on a succession of small successes over time. 
Theologians say that grace presupposes nature. If the spirit is not receptive, the soul cannot be 
saved. The same goes for assessment- the credibility of, and therefore the commitment to, 
assessment by faculty, staff, and students must be built over time. All groups must learn 
gradually that they derive benefits from assessment and that assessment will not be used to 
punish them. This last comment reveals my oias, and one that was made a first principle at 
Plattsburgh, namely, that assessment would be us d exclusively for diagnostic purposes — to 
improve performance and not to penalize. The purpose of this principle was not to make 
faculty or students comfortable with assessment, but to win their commitment to assessment as 
a way of improving performance. We wanted candid and open, not closed and defensive, 
responses to the penetrating and probing questions involved in assessment. 

The commitment of faculty, staff and students is critical to the development and 
implementation of a comprehensive assessment program. Even if a good program could be 
bought off the shelf, and it can't, the full participation of faculty is essential to success. 
Developing a sense of ownership through participation is the best way, not only to defuse the 
opposition of faculty but to win their continuing support and creativity for assessment. 

Assessment had a long history at Plattsburgh before I proposed a value-added program 
in a speech to faculty in the fall of 1984. We had begun an "Outcomes Project" in 1974 which 
developed gradually toward a full-blown assessment program. The project began with student 
surveys for freshmen and seniors. Freshmen indicated their expectations for intellectual, 
social, personal, educational, and professional growth. Graduating seniors used the same 
categories to assess their perception of gro* th during their college years. Both questionnaires 
also probed student perceptions of their abilities on a ran 0 e of intellectual and social skills and 
their intellectual and creative interests. Each year the resulis of both surveys were summarized 
by the academic majors of student respondents compared to the college-wide average and 
sent to all academic departments for comment. 

This information stimulated revisions in department curricula, in the college's general 
educational programs, and in campus services for students. This effort has produced a sizable 
longitudinal data base that remains in use today. Experiments were also tried with specialized 
and standardized tests in basic skills, general education, and academic majors. 

Without that long and largely positive experience, faculty approval of and, more 
important, participation in the value-added program would have been unthinkable. That new 
program, begun in 1984, included the development of a revised "Academic Development 
Survey" based on the original Outcomes work. It also proposed administering ACT tests to 
freshmen and juniors. Transfer students from community colleges also would take ACT tests, 
which would allow for comparison between native freshmen and transfers. Finally, it was 
intended that all students would take a nationally-normed exit exam, such as the Graduate 
Record Examination or the National Teacher Examination, to measure the degree of student 
learning within the academic majors. 
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It was the search for more appropriate examinations in the academic majors that led to 
the submission and approval of a grant from the Fund for the Improvement of Post-Secondary 
Education (FIPSE) to develop comprehensive assessments in academic disciplines. This 
project was conceived and led by Tom Moran at Plattsburgh. Pittsburgh's FIPSE project 
involves four academic departments in five SUNY colleges of arts and science. Faculty teams 
in each of the disciplines with representatives from each of the colleges will develop 
comprehensive programs for each of the four academic majors. The assessment programs for 
each of the disciplines and the experiences gained in the process will be shared widely in the 
SUNY system. The hope is to expand such consortia arrangements into other categories of 
SUNY campuses and to develop similar assessment programs for other academic majors. 

A second grant from FIPSE, this one to Fredonia, is being used to develop a 
comprehensive assessment of the general education program at that college. The Fredonia 
project is faculty focused. Campus committees, composed overwhelmingly of faculty, hr^e 
developed creative ways of measuring the educational outcomes of Fredonia's developmental 
general educational program. The project will evaluate student growth in critical thinking, 
reading, writing, problem solving, cultural awareness and social values through the use of an 
elaborate set of specially designed tests. The project will test incoming Fredonia freshmen and 
matched samples of juniors, and also a comparable group of freshmen and juniors at other 
institutions. 

Since Fredonia is a member of the Plattsburgh Consortia, the two projects in tandem 
offer the possibility of developing a comprehensive assessment approach that would include 
basic skills, general education, and specialized education in the major. Such a project could 
include the ACT Student Survey administered in all SUNY institutions every three years. 

The Fredonia and Plattsburgh projects represent only two examples of the assessment 
efforts in State University. Two system-wide committees chaired by myself as Provost, have 
recently been appointed to encourage assessment within SUNY. A Committee on Assessment 
is composed of experts on assessment from all categories of SUNY campuses. A Committee 
on Institutional Performance with Trustees, Presidents, and Students will consider the broad 
policy issues involved in assessment. 

Though we have not moved as far and as fast on assessment in SUNY as some of us 
would like* the system has developed a considerable potential for assessment while avoiding 
the major pitfalls. The approach to assessment in SUNY has been inductive rather than 
deductive. The movement for assessment has been centered on the campus and not in the 
capital and has been built on internal commitment rather than external compulsion. The 
measures used to assess are suited to the multiple outcomes of undergraduate education and 
avoid simplistic statistics and single scores. Assessment is used to improve and not just 
measure performance of schools and students. It has encouraged the emergence of a new 
notion of quality that combines excellence with access and assesses institutional performance 
in relation to institutional mission. SUNY has not done a 1 ! the right things. But what it has 
done, it has done right, in the right way, for the right reasons. 
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National Perspectives on Assessment 



Carol M, Boyer 
Senior Policy Analyst 
Education Commission of the States 
Denver, Colorado 



Introduction 

The state-mandated assessment of student outcomes has emerged as an unmistakable 
feature of the higher education landscape, with growing numbers of states requiring their 
colleges and universities to develop and implement campus-based assessment programs. More 
significantly, growing numbers of states are signalling their seriousness about the issue by 
linking the responses of individual institutions to eligibility for incentive funds or other tangible 
state rewards. What I am going to attempt to do (for the next hour or so) is to provide a 
coherent framework for thinking and talking about state-based approaches to assessment. 
More specifically, I plan to do four things: 

(a) set a broad context for a discussion of state-based approaches to assessment; 

(b) describe the purpose of and motives for conducting a series of five Education 
Commission of the States (ECS) case studies of state-based approaches to 
assessment in undergraduate education; 

(c) discuss several overriding "themes of implementation" that seem capable of 
summing up a great deal of experience — and literally reams of interview notes and 
file drawers of related case-study materials; and 

(d) draw some lessons learned about state-based approaches to assessment— some 
lessons in common for state and institutional implementation of assessment. 



Setting A Broad Context 

In the last 10 years or so there has been a slow but perceptible shift at the state level 
from concerns about financial accountability to concerns about academic accountability. With 
this shift has come a new emphasis on reviewing academic programs as well as linking state 
budgetary procedures more directly to institutional missions and articulating institutional 
missions more clearly. By the mid-1980s, improving the quality of undergraduate education 
was beginning to emerge as a political as well as edr ional issue. The series of now-familiar 
national reports issued in late 1984 and early 1985 (NIE, AAC, Bennett) mirrored the concern 
about K-12 that had followed the publication of A Nation at Risk and caught .he attention of 
the higher education community and of governors and other state leaders across the country. 
At about the same time the media began calling everyone's attention to students' declining test 
scores and their inappropriate preparation for entry-level positions after c allege, so strong- 
minded governors in a few states picked up on these concerns and led national efforts about 
what states and state leaders can do to improve undergraduate education. State-level 
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discussions about improving undergraduate education are now framed by their reports. 
Regional and professional accrediting agencies also started getting into the act, becoming more 
outcomes oriented in their approach (e.g., revised Southern Association of Colleges and 
Schools (SACS) criteria). 

In the past 2-1/2 years or so, then, the context vnuwi which discussions about "state- 
level reform of undergraduate education" take place has changed and it has changed 
dramatically. A few examples: In late 1985, ECS surveyed state-level academic officers about 
state initiatives to improve undergraduate education and found a "diverse array" of state 
initiatives directly tied to quality improvement. The initiatives were classified into 21 different 
categories, covering subjects as diverse as admissions standards, high school/college transition, 
"value-added" assessment of student outcomes, alternative resource allocation and funding 
mechanisms, special incentive funding for undergraduate education, and blue-ribbon 
commissions on higher education. About a year later (January 1987), ECS again surveyed 
state-level academic officers and found that concern about improving undergraduate 
education was increasingly being translated into concern about assessment. We also found that 
state leaders expected interest in assessment to increase in the coming year or two. And that's 
happening: 2-1/2 years ago, only a handful of states had formal initiatives labeled 
"assessment." Today, some 12-15 states have some sort of assessment initiative in place, and 
twice that number are seriously considering assessment. 

It is important to ask why all this interest in assessment, especially since it was 
essentially a footnote in most of the recent national reports. Let me suggest five reasons: 

(a) Assessment can be a powerful lever for institutional change in accomplishing all 
those other good things called for in the national reports. In fact, many of the issues 
of implementation raised by discussions of assessment are equally applicable io 
almost any type of management or program intervention. 

(b) Assessment can also be a powerful institutional tool for inducing faculty to 
reexamine their curricula. Many institutions report that reviewing curricula from an 
outcomes or assessment perspective and carefully thinking about possible sources of 
evidence, in themselves, helped to bring about needed curricular improvement. 

One institution, for example, reports that a significant impact of efforts to design 
"exit examinations in the major" is that it forced faculty to rethink the structure and 
intended outcomes of their curricula. As a result, changes in curriculum occurred 
even before any outcomes data were collected and disseminated. 

(c) Assessment attempts to break the cycle of people talking in circles and past one 
another, and to get down to issues that nobody has a good handle on. 

It's very easy to talk about improving undergraduate education in the absence of 
concrete information. Assessment provides a way of talking about improving 
undergraduate education in a very different fashion— one that involves further 
specification of the types of students involved, of the nature of the intended impact, 
and of the ways in which evidence of attainment is exhibited and assembled. 




(d) Assessment is also quite legitimate in maintaining public confidence outside higher 
education what has become an increasingly costly enterprise. 



In recent years, we have added institutions, programs, different kinds of student 
clientele — the list goes on and on. No wonder the general public no longer 
understands what higher education is about. No wonder that in an era of greater 
accountability and limited (sometimes declining) resources, state leaders are asking 
for concrete evidence that their resources are being well-spent and that college does 
make a difference. 

Higher education is also being sold on different grounds these days — linked to 
access, economic development, and international competitiveness. The public 
believes in higher education and therefore expects higher education to play an 
essential role in meeting the demands of a changing work force and society. This 
expectation, in turn, has led to a change in resource philosophy at the state level 
and to a change in what accountability means — from tell us what you did with the 
money to show us end performance or results. 

(e) And finally, when you come right down to it, the kinds of things outside agencies 
want are really not all that incompatible with what most faculty want except that 
they come from a very different perspective. 

Besides, it's important to acknowledge the historically important role of outside agencies 
in providing external pressure and focusing institutional attention on important issues. It is 
difficult to think of a single major reform in higher education that was not initially stimulated 
by external pressure, even though higher education eventually claimed it as its own idea (e.g., 
the G.I. Bill, affirmative action). Still, too much external pressure could destroy the entire 
improvement effort and turn it into accountability, so balan~^ is key. 



State-Based Case Studies 

In spite of the growing interest in assessment, the actual effects of state-mandated 
assessment remain uncertain. To investigate them, Peter Ewell of the National Center on 
Higher Education Management Systems (NCHEMS) and I recently conducted a series of case 
studies of assessment initiatives in five states: Colorado, Missouri, New Jersey, South Dakota, 
and Virginia. We wanted to gain a better understanding of how state-based assessment 
initiatives were unfolding — to determine what state leaders wanted to accomplish and what 
institutions were actually doing in response. We specifically chose emerging states (i.e., those 
with a relatively recent mandate) and states that differed markedly in terms of the structure 
and content of their assessment initiatives (e.g., governor's initiative in Missouri, legislation in 
Colorado, state agency initiatives in Virginia, New Jersey, and South Dakota). Additional 
differences among state-agency initiatives: 

• In Virginia, the state agency called for institutions to develop assessment programs 
that reflect their distinctive institutional missions but backed up that call with a 
Senate resolution. 
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• In New Jersey, the initiative was a creature of the Department of Higher Education 
building on their experience with the NJ. Basic Skills Assessment Program. Theirs 
is a centralized, top-down approach with faculty representation on statewide 
committees. 

• In South Dakota, the Board of Regents attempted to take a well-known institutional 
assessment model (Northeast Missouri) and impose it statewide. That didn't work, 
so this past summer they did a complete about-shift from statewide value-added 
assessment with common instruments to calling on institutions to develop their own 
approach to assessment. 

Our interest in conducting these case studies was prompted by several other motives as 

well: 

(a) The need for "rumor controP — there are several myths about assessment floating 
around, dangerous myths (e.g., that the only states that get involved in assessment 
are those with weak higher education systems . . . that states get involved without 
putting money behind it . . . that state-mandated assessment necessarily involves 
standardized testing or a "rising junior" component; and so on). It is important to 
put these myths to rest. 

For example, since the "assessment movement" began almost three years ago, only 
two states have introduced a program that includes a common standardized test 
(Texas, with its new basic skills assessment program, and New Jersey, with its 
comprehensive College Outcomes Evaluation Program that includes the development 
of a common instrument to assess "general intellectual skills"). The other states that 
use common standardized tests are Florida, Georgia, and Tennessee — and all three 
got into the business of assessment long before the assessment movement began. 

(b) We hoped for some generalizability in the findings from the five states and looked 
for patterns across states that others could learn from, as we were aware of a 
growing need for "good" policy. 

(c) Our interest in whether assessment is different from other kinds of state initiatives. 
Typical institutional reaction: "this is going to be just like a cost study or program 
review." State leaders are saying that assessment is a different kind of initiative — 
one in which they are trying to stimulate campus-based action on the part of their 
institutions. (Irony: State leaders say they want to empower institutions and foster 
formative behavior, but how do you mandate a formative process?) 

(d) The apparent tension between assessment for accountability and assessment for 
institutional self-improvement — and the notion that these are two completely 
different worlds (i.e., a zero-sum game) while state leaders are saying that the two 
go hand in hand. 

So here is what we did: We spent approximately one week on site in each case-study 
state interviewing governors, legislators and their staffs, state higher education executive and 
academic officers, and a cross-section of faculty and institutional leaders from public colleges 
and universities. Our principal question was: To what degree is the state's intent being 
translated into appropriate action at the institutional level? 



Themes of Implementation 



First, it is important to recognize that state contexts are vastly different and to note 
particular differences in the resource base (t*x + effort), in governance structure (e.g., "part 
of battleground" in Colorado) and in terms of the "climate" for higher education (e.g., 
education governors in New Jersey and Missouri). Such differences in state context interact to 
constrain and shape the types of initiatives that can be undertaken. They also provide 
"language" (e.g., "accountability" in Colorado, "folk implementation" in Missouri, etc.). 

Second, it is important to point out the differences across states in terms of state support 
for the initiatives (convenor and technical assistance as well as financial support). 

But, despite these differences in state context and state support, we did begin to see 
some patterns recurring across states. These patterns can best be summarized in terms of five 
"themes of implementation." Each of these patterns has implications for institutions currently 
deciding how to respond to a state mandate (see article in Change Magazine, July/ August 
1988). 

The salience of past or parallel experiences. Assessment is an unfamiliar exercise to most 
state leaders and policymakers, so they seek a "parallel experience" to guide action. For 
legislators, the obvious candidate is past action in the K-12 arena. We found many references 
in language (Missouri and South Dakota), but a downside in misunderstandings 
("accountability" in Colorado). The other possibilities were the prior experience with 
implementing similar kinds of programs (e.g., the New Jersey Basic Skills Program extended 
to COEP in terms of the process of execution and implementation, committee structure, who 
does what when, etc. Another possibility is the use of existing institutional models (e.g., 
NMSU as ready-made model in Missouri and South Dakota). 

We identified those lessons for institutions that relate to this theme: 

(1) It is inevitable that such models will be used, but it is important to recognize that 
this is going on. 

(2) Be aware that state leaders often intend these prior experiences more as "trial 
balloons" than as definitive approaches to state policy. 

(3) Recognize that state leaders can be talked out of it. Help them translate the 
experience and shape the assessment initiative (e.g., "educating the sincere 
amateurs" in South Dakota). 

The primacy of communication. Overall, the thorniest problems in implementing state- 
based assessment initiatives arise from unclear :ommunication and miscommunication. We 
found that virtually every kind of miscommunication that could happen, did happen. It is 
important to recognize that communication between state and institutional leaders is, by 
definition, political. Entry of the state into a previously academic dialogue can thus easily 
result in unintended polarization. It can also lead institutions to search for "hidden agendas" 
and consequently to misunderstand what state leaders actually want. Other communication 
difficulties have to do with the particular language used (e.g., "accountability" in Colorado, 
"nonanecdotal, quantitative" in Virginia), the lack of a common vocabulary, the timing of 
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communications (long silences in Virginia and Colorado, irregular patterns, etc.), and the 
particular forms of communication used (formal vs. informal, "folk implementation" in 
Missouri, lack of clear document trail, etc.). 

Lessons for institutions: 

(1) Avoid searching for "hidden agendas" and "seeing what you want to see" in the 
content of a state initiative ("fondest hopes, worst fears" phenomenon). 

(2) Initiate, don't just receive. Establish and maintain an ongoing dialogue with state 
leaders and policymakers rather than second-guess their intentions or jump to 
conclusions. 

(3) Communicate about assessment and related activities to the state through multiple 
channels. Visibly use assessment data, both within and outside the institution, to 
support and to communicate major policy decisions. 

(4) Start communicating assessment results through existing statewide data collection/ 
reporting mechanisms, even if assessment results are not explicitly required. 

(5) Be realistic in communicating institutional concerns about multiple and 
inappropriate state-reporting burdens. 

(6; Communicate regularly with other institutions in the state about progress being made 
(and obstacles being encountered) in responding to a state initiative; share ideas and 
resources, address issues of mutual concern, and so on. 

Academic and political worlds. There are lots of differences betweer these two worlds. 
One area of substantial tension has to do with timing. 

Institutional timetables are long and conditioned by the particular rhythm of the 
academic calendar. For the most part, state leaders and policymakers don't recognize this 
(e.g., Virginia, Colorado). They are largely unaware of how long it may take for faculty 
governance structures to act or react. They don't understand that there may be long "down 
periods" in academic settings during which no substantial action can occur. Political timetables 
are tied to terms of offices. Political leaders are concerned about the delivery on promises 
made during campaigns (e.g., even in New Jersey, Governor Kean is running out of time). 

Other timing difficulties arise through different "rhythms" in the academic and political 
calendars (e.g., biennial budget "window" was the only opportunity for action in Virginia). 

Lessons for institutions: 

(1) Be aware of the many constraints on political timetables and be realistic (but not 
righteous) in communicating the constraints on your own. 

(2) Become i ware of the other inherent tensions between the academic and political 
worlds and act accordingly — develop a sensitivity to the constraints placed on 
behavior in the two worlds. 



The "press to test." Regarcuess of the actual shape of a state assessment initiative, we 
found enormous pressure of discussion toward the alternative of common standardized 
achievement testing. For policymakers, this is often the first, visible posed alternative (though 
not necessarily what they want to make happen). For institutions, this is what they are "sure" 
has been state's agenda from the very beginning. Where common testing is an explicit part of 
a state initiative, it tends to absorb more than its due share of attention and energy — and 
sometimes the lion's share of resources. 

Lessons for institutions: 

(1) Be aware that common standardized achievement testing may initially be proposed 
by state leaders as a "trial balloon" and respond accordingly. 

(2) Don't let issues about common testing so politicize discussions that they obscure 
those other components of a state initiative that are consistent with institutional 
purposes (e.g., faculty member in New Jersey: "You can't stop it, but you can shape 
it"). 

A "train on its own track. " A particular implementation dilemma encountered was the 
tendency to isolate the assessment initiative from other existing state-level programs and 
reporting requirements such as mission and program review. Granted, our case-study states 
were ones that had explicit initiatives, but pursuing assessment as a distinct and identifiable 
activity can place heavy, often contradictory burdens on institutions (e.g., Colorado H.B. 1187 
provisions). 

There is also some fear that assessment will become a separate bureaucratic enterprise, 
with a separate set of compliance requirements and with no direct tie to curriculum or 
improvement. 

Lessons for institutions: 

(1) Consider taking a positive stance — that h t getting out ahead of the state-mandated 
initiative with a range of locally designed programs, then "selling" these programs as 
consistent with the evolving state mandate. (Several "wait-and-see" institutions said 
next time they would be proactive.) States seem willing to listen to a wide range of 
local options if intelligently sold, but states (and rightly so) may have little patience 
with lack of institutional response without "clear" direction. 

(2) Whenever possible, develop assessment programs that will gain reasonable faculty 
ownership and that are designed primarily to meet local institutional needs. 

(3) View institutional assessment plans as evolving documents; modify and improve 
them as appropriate. 

(4) Visibly use assessment results to inform institutional change and academic 
improvement. 

(5) Seek additional outside resources; compete with other institutions for explicit state- 
based resources dedicated to implementing assessment programs, and for funds 
from FEPSE and other agencies or foundations. 



[A word of caution: Lessons such as these often sound simplistic and common sense. 
This reminds me of a colleague who summed up her review of the organizational 
literature on leadership and management behavior with this simple phrase: "Don't do 
stupid things!"] 

These themes were present in varying degrees in all five case-study states, and each 
colored and shaped the manner in which the assessment initiative was played out. What about 
the "bottom line" of institutional response? We found that, * general, institutions tended to 
respond in one of three ways: 

© A few institutions cho e initially to resist the initiative. That generally proved 
unsuccessful and in doing so the institutions tended to provoke direct action on the 
part of state agencies to achieve compliance. 

• A second category of institutions, by far the majority, took a "wait-and-see" 
attitude, reacting only to those things explicitly asked for by the state (and only 
when asked), and sticking closely to what they thought the state was looking for. 

• A final category of institutions took assessment as an opportunity to accomplish the 
local agenda (e.g., South Dakota State University, Metro State in Colorado, 
Northwest and Southeast Missouri). But doing so requires active leadership at the 
institutional level and may require "selling'' the local initiative as consistent with 
state guidelines (e.g., "declare victory"). It may also entail the risk of getting too far 
ahead with the possibility of a bureaucratic "cutoff" (e.g., Metro State). 



Some Lessons in Common 

In closing, let me suggest the following practical guidelines or lessons in common for 
both state and institutional leaders. 

• Capitalize on existing information. Most colleges and universities collect far more 
information and know a great deal more about what happens to their students than 
they think they do. There is information all over the place — information that is 
neither coordinated nor pulled together centrally, largely because there is no 
incentive to do so, 

Here is where the state can come in, by creating the incentive for pulling together 
existing information. Registrars' offices, testing centers, placement offices, 
counseling centers, and departmental offices are all good places to look for existing 
information. 

Start by taking an inventory. Compiling information from various sources to reveal a 
composite picture of student experience is a good first step toward implementing 
assessment. 

The state-level equivalent has to do with what institutions know about themselves. 
One of the roles a state can play is that of convenor, by providing the opportunity 
for individual institutions to learn from one another, to share ideas and resources, 
and so on. 
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Utilize existing "points of contact" with students. Take a look at those places where 
you are already collecting information for some other reason and piggyback 
assessment on top of that. 

As an example: an institution that was assessing writing — and doing it 
developmental^ The students were required to write an essay, which was graded 
holistically, when they entered the institution and another essay at the end of their 
freshman year. The interesting twist was the topic of the essay: "What are your 
expectations for college life and what are the major strengths and weaknesses of this 
place as a learning institution?" They got some very interesting stuff quite aside from 
the fact that they were able to assess students' writing. 

What is the state-level equivalent? Take a look at the potential of existing 
mechanisms such as planning or program review* Look for places where you are 
already collecting data from institutions. And look for ways to piggyback some 
outcomes or assessment data on top of those mechanisms. 

Create a visible center for assessment activity. Assessment can make a powerful 
statement about an institution's concern for the development of its students. To be 
effective, this statement must be organizationally visible. 

At the same time, there needs to be an organizational home to integrate the 
assessment i iformation and to provide some technical assistance to faculty and 
departments. 

The most successful assessment programs designate a particular office or staff- 
supported faculty committee for housing and directing assessment activities across 
the campus. They fund it adequately, and they provide it with powerful channels for 
disseminating results both within and outside the institution. 

Experiment with pilo % projects. Don't go into the assessment business "whole hog" at 
first. Take the time to examine some of the unanticipated consequences of what 
might occur. Take the time to find out what works and what doesn't. 

There are some positive aspects of pilot programs quite aside from guarding against 
disaster. If you provide opportunities both for institutions (if you are the state) or for 
academic departments (if you are the institution) to experiment with pilot projects, 
you will have the opportunity of choosing those pilot projects very carefully to 
provide future advocates for an assessment program or initiative. 

If, as I think some of the best institutions in state systems have done, you encourage 
those who already have an interest in assessment, all of a sudden the state isn't 
telling the institutions what to do anymore, and the administration isn't telling the 
departments or the faculty what to do. It is institutions selling institutions (or faculty 
selling faculty) on the benefits of that kind of an endeavor. In short, bet on the 
strong. 

On any campus, assessment is a novel and controversial endeavor. Faculty will be 
apprehensive, administrators defensive, and students indifferent. Moreover, untried 



methods will be proposed, the consequences of which are unclear. Beginning with a 
series of varied pilot programs, using volunteer units and departments, allows a 
number of approaches to be explored simultaneously. And faculty in successful pilot 
programs can serve as powerful advocates for assessment across the campus. 

• Use the results in identifiable ways. No process of tuis kind is going to be taken 
seriously if the results aren't taken seriously. If it just becomes a compliance 
. exercise of collecting and reporting assessment information, then it is no', likely to 
result in any kind of change. 

There needs to be some mechanism that encourages institutions to build assessment 
information into planning or into budgeting in a visible kind of fashion — and in a 
way that is directly tied to institutional accountability. 

Assessment will not be taken seriously on or off campus if its results are not taken 
seriously. If a periodic report is all that is produced or required, and if assessment 
results do not show up visibly in decisions about curriculum, academic policy, and 
budget, then the whole process will fall short of realizing its full potential. 

In summary, developing an appropriate approach to assessment (at the state or 
institutional level) is an art that depends on a clear understanding of what is intended, 
information about available instruments and about the experiences of other institutions or 
states, and accurate diagnosis of the organizational and political climate. As in any other art, 
there is no single best way, but it is hoped that the re c ults of the ECS case studies and the 
lessons learned from them will inform state-based approaches to assessment in undergraduate 
education that are just getting under way. 
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Public Perceptions of Higher Education 



Edward B. Fiske 
Education Editor 
The New York Times 
New York, New York 



Pve been asked to address the question of American perceptions of the value of higher 
education. It's one of those topics that reminds me of the story they tell about Oliver Wendell 
Holmes who was riding on the old Pennsylvania Railroad and the conductor came into the car 
where Holmes was riding and started to collect the tickets. Justice Holmes reached into his 
pocket to find his ticket, started looking all over and began to panic. He finally got up and got 
his suitcase down from the rack over his seat and looked in it and finally got down on the 
floor, on hands and knees, and was thrashing around to find his ticket. As the conductor drew 
near, he realized who this was groveling around on the floor of the train, and was horrified 
and told Justice Holmes to get up, saying "I'm sure my superiors of the Pennsylvania Railroad 
can trust you to send in your fare when you get to your destination. " Holmes looked up and he 
growled back at the guy — "You don't understand the situation: the question is not whether I'll 
pay but the question is where am I going?" I feel a little bit like that on this subject. 

The first thing you have to say: there is no doubt at this point we value higher education 
and probably we value it as much as we ever have. As evidence of that, look at the 
admissions picture in the last couple of years, including the most expensive selective schools. 
People are lining up to pay $20,000 a year at the better colleges. I have to admit even though 
we write about this, we really don't understand what is going on. It's sort of the big unknown 
question. The admissions picture is against what the demographics hold; we certainly have a 
decline in the number of 18-year-olds. Yet even at schools that only cater to the traditional 
market, there is enormous pressure to get in. Partly, I guess, it's a response to the marketing 
colleges have done — very sophisticated about recruiting. 

I think one of the elements that has not gotten as much attention as it should have is the 
fact that now we have the first generation of college applicants where substantial numbers 
have college-educated parents and who grew up always expecting to go to college. What you 
see now is the children of people who participated in the vast expansion ot higher education in 
the 1960s — the years the State University of New York grew. Pve never seen anybody soin this 
out — Pd love to see colleges look at their own experiences here — what we know about the 
college education of students today and that of students earlier. 

So I think that's a factor. And then the most important thing is that people are buying 
up. People have decided that higher education is a good investment and they are trying to get 
into as good a college as they can and they are willing to pay and that's why ihere doesn't 
seem to be much elasticity in the pricing of the more selective private colleges. 

I think people also intuitively understand that the future is going to he different. As 
Adam said to Eve on the way out of the "Garden of Eden" — "we live in an age of transition." 
The days are over wnen you can go and get a college education and store up a certain amount 
of knowledge and that will last you over a career. I think people understand that and we are 
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now in a situation where it's more a learning to learn — you people have all heard this before — 
but it's getting across to people. People are scared about the future— they are nervous about 
the future— that's why education is going to be an issue in the presidential campaign. Not 
education per se, but it is a shorthand way to talk about the future and the world. There is 
probably one issue in the current campaign that is beyond Reagan, and education is a 
shorthand for talking about kids and the future when symbols of that are hard to find. People 
understand this. They understand that the United States is now in the kind of situation that 
Japan has always been in — at least since the late 19th century- Japan is a country about the 
size of California, has a little over half of the American population crammed mt( 1/3 of the 
country — two-thirds is uninhabitable. It is a country with no natural resources. So Japan is a 
country that has always had to live by its wits. It has nothing going for it except what comes 
out of its heads. And we have always been able to tolerate a certain i^mount of inefficiency in 
education because we have timber, farmlands, oil — natural resources that we can fall back 
upon. But the ecological frontier is over. We, like Japan, have to from now on live by what 
comes out of our heads, live by our wits. I think people intuitively understand this. 

The third thing that you see is the emphasis on quality. I said people were buying up. 
They see education as an investment. They want quality, they value quality and, of course, 
colleges understand this. Quality becomes the central marketing tool for recruiting purposes, 
and as a matter of fact, I think by now the significant differences in American higher 
education are not public-private but good-bad. It's high-qrlity institutions versus low-quality 
institutions in both public and private sectors. If you look at the real issues, they tend to sort 
out this way. I guess in some ways this preoccupation with quality has led us to why we're here 
today — why accountability and assessment and productivity are a set of issues that have 
arisen. 

I thought the analogy that Carol Boyer made about the K-12 school reform produced 
some significant things. This began in the late 70s with the minimum competency movenK.K 
which was a fascinating phenomenon — in the space of about three years (1975-1978) — we went 
from C to 35 states which had minimum competency programs. And there the basi'; problem 
was employers saying the schools are turning out students who can't read and write. You've 
got to shore up the bottom. But people began to say that s not really enough. In the late 70s 
and early 80s, it was articulated in A Nation at Risk and a rising tide of school reoorts. The 
current school reform movement was aimed at the higher levels — we shored up the bottom and 
now there was a need to chore up the higher thinking skills. So that's what A Nation at Risk 
was about and that's what we have been looking at for the last five years. 

As you are well aware, reform has gone through several waves. And everybody has their 
own way of defining these waves. But I would do it like this. The first wave did what the 
political process could do. It defined the inputs and mandated the outputs. It sort of put the 
parentheses around education. This was natural because the political process — the original 
impetus for school reform — had nothing to do with educators. As a matter of fact, it was often 
brought about by the opposition of educators. Essentially a coalition of businessmen and state 
political leaders did what the political process could do: they raised the money. You can pay 
teachers' salaries, you can have a merit pay plan or whatever — and you can also say on the 
other end here are the goals — you have to take so many courses to get a diploma from the 
state of New York or you have to take this test or v; ; .itever. We went through the process and 
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most states did essentially what you could do in this sense. But that left undone what happens 
ia the classroom. Ultimately you have this situation where the door slams behind the teacher 
and the student and the governor is not there and the president of Xerox is not there — it's the 
teacher and the kids. So people realized we've done the parentheses but somehow the middle 
has to be addressed. 

The second wave then focused on the teaching-learning process. People started talking 
about the teacher. We had learning reform coming out of teacher certification. That filled in 
the middle. Even so, five years later we still realized that not enough has happened. I was 
down at a White House show-and-teil session to celebrate the fifth anniversary of A Nation at 
Risk. On this occasion people got together to see what had been accomplished. There seemed 
to be a pretty general consensus that we had wrung out of the current system what ^-,uld be 
wrung out of it. It did improve standards and got people to fit into the system and do better, 
try harder, and take more basic courses. Maybe you can only push this so far. Maybe you 
have to go back and start thinking about some fundamental changes in some fundamental 
structures. That's why we are having a restructuring thing going on. The point is that we have 
a rather incomplete school reform movement which is going to continue. It may well be we will 
see the first complete change in the system in our country. In talking about higher education, 
one of the reasons why we are here today is that the school reform movement is bumping up to 
the higher education level. 

I'd like to make a footnote on a point that was made this morning about external 
pressure. That is, the GI bill was brought about not by the education establishment. As a 
matter of fact, it's not known by a lot of people, but higher education lobbied against the GI 
bill. It's hard to believe, but the reason, I think, was the GIs would change the campus — you 
would have older students, wives and babies, and the colleges were not geared up to this — and 
it just would not be a civilized place. The higher education establishment, the DuPont Circle of 
the day, went down and lobbied against the GI bill. The footnote here is that a lot of things 
that affect higher education are not only external to it but done for noneducational reasons. 
The GI bill was not an effort to help education but was a baby-sitting plan to help returning 
veterans and a way to reintegrate them into the workforce. The Elementary and Secondary 
Education Act was not an act to improve education but was an anti-poverty program. The 
Middle Income \ssistance Act in 1978 was not a student assistance act but was middle income 
assistant \ If you follow the federal legislation which has had a big impact on higher 
education, you usually find it was done for reasons that have nothing to do with education. In 
some ways this is the ultimate compliment and goes back to my topic — do Americans value 
education? It's so intrinsic that whenever we have a problem we look to education as a vehicle 
to solve it. 

School reform is bumping up to the college level. It was certainly intensified by Bill 
Bennett who came charging in ready to do battle. At his first news conference, he said 
American colleges were ripping students off. And that was the stereos-Florida vacation 
speech. He came in and part of his plan was to go after the colleges. We are going to miss 
him. As Mark Russell said when Nixon left, "Now I have to go back to writing my own 
material." 
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Anyway, Bennett did have an instinct to go to the two places where colleges are 
vulnerable. The first is on costs. I think the vulnerability is not just the rising costs — you 
probably can make the case that education is a little bit more expensive than general inflation. 
But the explanations colleges gave for the rising costs were only loosely related to the reality. 
Every year we get these statements from college presidents. Because we are now a labor 
intensive industry and we are facing soaring costs much greater than other sectors, and the 
need to catch up on faculty salaries and maintenance, and depending on the year, it is energy, 
libraries, computers, you fill in the blank- I start reading these and ask, "What's this year's 
blank going to be?" The implication is we are just sitting here, and our costs are going up, so 
we are just passing these costs on to you. Well, it took me a long time to figure this out, 
probably longer than it should, but it didn't square with reality. I finally figured out that 
pricing is more market-related than cost-related. Bowen's point that colleges get what they 
can and spend up to it is true. Basically 5 colleges price themselves on what the market will 
bear and then do a lot of discounting — called financial aid. You don't compete by price. 
Sectors compete by price. The Ivys have one price, the Amherst, Williams, Wesleyans have 
another price. But clusters of colleges compete. If you are in the Kenyon-Oberlin cluster, you 
can't be more than Harvard, Yale, and Princeton. But there's not competition on price among 
colleges. So you charge what you can get and then discount. People began to figure this out 
and the problem was one of candidness. Bennett has a point here and colleges are vulnerable 
on the cost issue. 

The second area where colleges are vulnerable is the curriculum. This is a little trickier 
but the curriculum does lack coherence. If there is coherence, the general public and education 
writers do not understand it very well. Let me take a running jump on this one. I first started 
writing about curriculum back in -the mid-70s when Harvard started its discussions of the core 
curriculum. Because Harvard was doing it, everybody else did it. Basically what you saw was 
the recovery of distribution requirements. And I remember writing this and had lots of 
interesting conversations with Henry Rosovsky. Beyond that, it was hard to find interesting 
ideas about cur .culum. All people seem to go back to the Chinese menu approach — so many 
credits in the humanities, so many credits in sciences, etc. I got bored by the subject because I 
couldn't find people doing interesting things. Then in the early 80s, other colleges were talking 
about curriculum again. In looking back on dl this, I figured out what had happened before. 
What took place in the curriculum reform of the 1970s had nothing to do with academics or 
higher education. It was political. Colleges were reasserting theli authority over the curriculum 
that they had ceded to the students in the 60s and early 70s. It was simply grabbin c i nrr'er 
back — grabbing the control back. If you are going to do that in a way that will noi upset the 
political turf within the faculty, you take something from everybody. It was the most neutered 
way you could bring a little coherence to the curriculum because no one lost anything. The 
faculty collectively regained control of the curriculum, except at places like Brown. Then 
colleges began to realize that the distribution requirement approach is a little mindless. It 
deals with more structure than content. 

We began to see colleges asking the question — what does it mean to be an educated 
person? As far as I can see, what people came up with next was a skills add-on. Distribution 
requirements v/ere there. So we next added skills — writug, calculation, computer use. And 
there was a set of content add-ons — you have to know something about a non-Western 
culture, as recently seen in the debate at Stanford. Or gender studies or others. Basically, 
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that's the curricular reform I have seen. There are some schools that are going beyond that 
now— I just haven't had a chance to look at them. But the basic question is, what does it mean 
to be an educated person? Bennett's instincts were right. Colleges really haven't thought this 
through and he went after them on it. 

The last and possibly most important point is that we are in an age where productivity 
and accountability are in the air. We realize much of this is driven by economics. The post- 
Sputnik school reform was a response to an external military threat. The current school reform 
can be viewed as a response to an external economic threat. As a matter of fact, Ernie Boyer 
said that ™hnt we should do is get the Japanese to shoot a Toyota into orbit. We can have a 
mow visible symbol of the problem we face. So we are concerned with productivity— get our 
act together, tighten up— the auto companies are reorganizing. And the obvious question is, 
why does higher education claim it's the only industry that is immune from this? We don't talk 
about productivity in higher education circles — it's a no-no! 

You are not supposed to talk about it — the terms of higher education arc different- -it is 
a different enterprise. But that's not clear to most people. It may take on special 
significance — because along with entertainment, higher education is the one area where 
Americans dominate this world in terms of exports and a favorable balance of trade. It is an 
important industry. Once people raise this question, they look around and see some obvious 
areas for improvement. Like the summer downtime. I realize there are big problems of what to 
do with the campus in the summer but to the outsider, it really doesn't look coo smart. Imagine 
Ford closing down one of its assembly plants for the summer not because of a drop in sales but 
because that's the way they want to do it. If you look at the school year, I think you could 
probably make a case that the length of the school year in American colleges is inversely 
proportional to the tuition. People wonder why in the fkst week in May they are reading a 
story about Columbia's graduation. What a life! 

And then maybe the most compelling reason is the whole question of the internal reward 
system — the teaching/research issue. Here I would point you to the data that Boyer got out of 
the College survey. He finds that a lot of people out there are doing research who would rather 
be doing something else. Ninety percent of the serious research is done by 10 percent of the 
faculty. Everybody knows that many are grinding away on useless research to make a big 
resume, to get tenure at a place where you are supposed U be teaching anyway. The 
underlying theme here is that maybe the values of the academy are out of sync with the values 
of the society supporting it. People are very concerned about the quality of teaching. You look 
at reward systems where teaching doesn't get rewarded. I realize this is ^ omplicated issue 
and there certainly is a point at which good teaching and good research have a symbiotic 
relationship. But maybe not in every institution. Boyer talked about a system where you don't 
have to contribute new knowledge to your field but you have to demonstrate that you are up on 
what others are doing. Maybe we need a new definition of academic competence for people 
who aren't going to be generating new research. Certainly, it is within the capability of 
individual institutions to alter ways of rewarding good teaching — salary bonuses will go a long 
way; sabbaticals; various other kinds of recognition. The basic point Fm making is that I'm not 
sure on the teaching/research issue that the values of society are reflected very closely with the 
values of the academy. 
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Then you get to the question of value-added testing. I realize that it's very difficult to 
define some of the goals of the academy. It has been suggested, for example, how do you 
measure basic skills or subtle things, skills for the future like historical sensitivity or literary 
sensitivity. I should remind you that there are places where this has been looked at, including 
the judicial system. Columbia University about 15 years ago sued a student for nonp yment of 
tuition. The student filed a countersuit, saying "I didn't owe you any money because you didn't 
make me wise." In his legal brief, he cited statements in the catalogue, pronouncements by 
senior administrators and inscriptions on campus buildings, saying four years at Columbia 
would imbue you with wisdom. The judge dismissed the suit on the grounds that there wasn't 
any precedent in common law. But the judge also couldn't avoid the temptation to throw in a 
zinger of his own: he quoted Alexander Pope that a little knowledge is a dangerous thing. It 
was a fascinating decision to read. 

Having appealed for more emphasis on productivity (this is what the public is asking 
for), let me also argue the ether side. One of the things which strikes me as a journalist is the 
extent to which college administrators pride themselves on how worldly they are, how effective 
they are as managers. So you see colleges present themselves with marketing analogies. This is 
fine up to a point. But the culture of higher education is very different from that of industry. 
It's why businessmen tend to be disasters as college presidents. It is essentially a political job 
where you need the power to persuade, not make a decision. It's a culture that values process 
more than results. The process itself is the goal, not necessarily any resolution cr decision. 
There's also a point at which the market analogy breaks down. When buying a bar of soap 
from Proctor and Gamble or a Chevrolet from General Motors, for the most part, it's between 
you and them. You buy a product. But when you purchase higher education, there is a third 
party involved — that is the public. All higher education is subsidized and the country has a 
stake in the outcome of that purchase. What Carol Boyer was saying about the openness of 
state education officials to be persuaded is true. The public will tolerate the differences 
between higher education and other industries and can be persuaded of them. The problem is 
that you in higher education have to tell them what you are doing, you have to persuade them 
better. Defining the new goals or finding a new definition of what it means to be educated or 
to go back to problems of the curriculum — it's hard to find people in education, much less 
whole faculties, who can articulate for me what they think it means to be an educated person 
in the 7lA century. So a lot of the problems do have to do with communication. If you can 
communicate something that presumably means you know what you are trying to say. That is 
the message or the missing element. Higher education has to think through better what it is 
about and then all the other issues fall into place. 
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We are delighted to be here with our colleagues in New York to participate in this 
discussion- We have had an opportunity to speak to a number of groups around the country 
describing some of the assessment activities taking place in our state and on our campus. This 
afternoon we hope to share with you information that will improve your frame of reference 
concerning assessment and give you ideas for approaches that you might fake, as well as some 
things you might avoid, as you plan to undertake assessment activities. 

I suspect that most of you know more about Tennessee from the perspective of Opryland 
and Dollywood than from your knowledge of higher education in the state. As you will note, 
we do feel pretty good about some of the developments in higher education in Tennessee, 
particularly in the last eight years. We have two systems of higher education in Tennessee, 
including a system under the jurisdiction of the Board of Regents which includes 14 community 
colleges and 6 four-year institutions. The University of Tennessee is governed by a Board of 
Trustees aid has four campuses including the Knoxville campus, the graduate and research 
institution of the state of Tennessee. We have 19,000 undergraduates and 6,000 graduate 
students. The total budget exceeds $250 million annually, including approximately $60 million 
in sponsored research funding. 

The emphasis of our comments today will be upon assessment programs which are 
focused upon undergraduate education but also have some implications for graduate programs. 
The Tennessee performance funding program began during the mid-1970s vhen enrollment 
pressures were beginning to ease. In fact, we were projecting a period of level enrollments and 
anticipating some declines. As was the case in a number of other states, Tennessee higher 
education officials began considering qualitative initiatives. A number of approaches to 
program improvement were discussed, and the earliest of these was the performance funding 
program which was implemented with pilot tests during the late 1970s, The earliest version was 
begun on all campuses between 1979 and 1982. The first five-year performance funding 
schedule was put in place in 1983. The program began with mixed reviews, and some 
campuses encountered resistance. The faculties of the colleges and universities in Tennessee 
felt that they could appropriately assess quality without such a set of state-provided 
guidelines. However, over the period of phased implementation of the program, we observed 
greater acceptance of the program as well as improvements in the schedule itself. We 
completed the first five-year schedule just this year, and we are now moving into the second 
five years of performance funding. The new five-year schedule includes fiv2 standards that 
must be met if an institution is to earn all of the funding for which it is eligible, and for the 
first time provides an optional standard which allows an institution to earn additional points 
and the associated funding. If an institution completely fulfills all requirements of the first five 
standards, an amount equal to 5 percent of the institution's total instructional budget will be 
added to the institution's annual appropriation. For UT-Knoxville next year, the amoui;* of 
money for which we would be eligible (if all points are earned under the first five standards) 
would be approximately $4.5 million. 
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The first standard is program accreditation. Institutions are given credit on the basis of 
the percentage of accreditable programs that are fully accredited. That particular standard has 
declined in importance in the new schedule because, over time, more and more programs 
throughout the state have been accredited. During the first five years of the schedule, this 
standard worked as a strong incentive for program accreditation, particularly for the 
community colleges and smaller four-year institutions. Twenty points are available for this 
standard. 

The second standard provides for the testing of graduating seniors in the content of their 
major field (and for graduate institutions' external reviews of master's programs). This is a 
component that has remained constant in terms of relative weight in the point schedule from 
the first five-year cycle to the second. Up to 30 points may be awarded on the basis of 
performance and improvement in performance using standardized examinations in the major 
field (professional exams for accounting, nursing, teacher education, etc.) or local exams 
developed at the departmental level. The local exams are subject to external review before 
they are approved and administered. We have seen many academic departments across the 
state develop their own local exams, giving the faculty an opportunity to develop examinations 
in response to curriculum goals. 

The third standard is the testing of seniors in general education. The focus here is on 
two factors: the total scores earned on the ACT-COMP test by the institution's seniors 
compared with those of their peers, and the score gain registered by seniors compared with 
their performance on the COMP as freshmen compared with gain scores of students at peer 
institutions. Institutions may earn up to 20 points on this standard with up to 10 points 
available for each factor. 

The fourth standard, one that has proved to be significantly important over time at most 
institutions, involves surveying alujani and dropouts concerning their perceptions of program 
quality. The information gained from surveys is examined for improvements and also 
compared with norms and contributes to a total of 15 points under the new schedule compared 
with 10 points under the original schedule. The committee recommended the additional points 
as the 1988-92 schedule was developed because of a strong belief that the survey information 
had played a major role in program improvement. 

The fifth standard, which provides up to 15 points, recognizes improvements made by 
the institutions based on findings from the assessment activities of Standards II, III and IV. 
This standard has been referred to as the planning standard. The institutions use information 
from a variety of assessment programs to improve instructional and support processes and 
thereby improve student outcomes over time. For many of us, this has been the most 
significant standard because it has enabled us to integrate outcomes assessment with ongoing 
planning activities. Trudy Banta will talk about the steps taken in that regard at the University 
of Tennessee, Knoxville. 

In the new schedule, in order to encourage research in the assessment area and to 
further improve the overall framework for performance funding, we added a standard for 
pilot-testing new instruments. This optional standard provides up to an additional 10 points 
over the five-year period for research and development in the assessment area. This standard 
provides an opportunity for institutions to continue their exploration of assessment programs 
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and gives credit to pilot-testing new approaches to assessment, reporting on findings, and 
demonstrating how these approaches can be used in institutional decision-making. 

Question from audience: "Can you tell us how you redistribute the extra funds received 
among the departments? If you have a department that didn't make it and one that did, what 
happens?" Response: In our annual budget hearings and in our planning, we use information 
gathered from the assessment activities in several ways. A department can justify additional 
resources on the basis of its demonstration of quality using a variety of outcomes sources. A 
department can also use the outcomes assessment information to point out the need to 
strengthen the curriculum in a particular area and justify additional staffing or more 
opportunities for faculty development and thus request resources in support of those needs. 

The performance funding schedule itself may not have been as important in some of our 
successes as some other factors that have made the program workable over time. Here are 
some of the strengths of the program: 

1. The program is voluntary. An institution does not have to participate in the 
program. In our case, I think we would have made a serious mistake if we had not 
participated because we have found that we have been able to use the information 
in our ongoing planning, policy-making and decision-making framework. I must 
emphasize, however, that our participation was voluntary. We could have decided 
that we didn't want the $4.5 million this year. 

2. Participation produces a substantial budgetary supplement. The program is not 
punitive. It was put in as a true supplement. Obviously, that has had a substantial 
impact on the overall reacticu to the program. The initial supplement provided up to 
2 percent of the instructional budget. Later it moved to 3 percent and is now 5 
percent. These increases in the relative weight were achieved after deliberations by 
a committee of institutional lepresentatives, Board staff and Commission staff. This 
group believed that the performance program had been sufficiently successful to 
warrant an increase in the funds distributed through the program. That unanimous 
support for the distribution of additional resources through this program as opposed 
to distribution by the funding formula would not have been present at the time the 
performance funding concept was initially introduced. 

3. The program was phased in gradually, effectively over eight years. 

4. The criteria have flexibility. It rewards absolute performance, improvement and, to 
:>orae degree, participation in the process itself. 

5. There is not competitioo among Tennessee Institutions for funds. We are not 
competing with each other. We do compete nationally with our counterpart 
institutions as we compare our students' performances on particular examinations, 
but we do not compete with other institutions within the state in earning points under 
the framework of the standards. 

Carol Boyer mentioned the new accreditation standard of the Southern Association of 
Colleges and Schools related to institutional effectiveness. Much of the emphasis of the new 
standard reinforces the performance funding program in Tennessee and has encouraged other 
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states to look at similar approaches to assessment- The new standard stresses planning and 
evaluation with the involvement of both faculty and administrators. It stipulates that 
institutions must establish a clear purpose and mission. Goals must be established consistent 
with purpose, and the progress toward achievement of goals must be assessed with the 
information gained used to improve effectiveness. All of these aspects of the new standard are 
recognized to at least some degree in the Tennessee Performance Funding Program. 

About 18 months ago, my colleague Trudy Banta and I surveyed the institutions in 
Tennessee to get some reaction to performance funding among our colleagues. The timing was 
not an accident. We felt we could use this information as the schedule for the second five-year 
cycle was being developed. I would like to share the highlights of the survey results with you. 

Our findings concerning responses to Standard I, accreditation, indicated that 12 of 22 
institutions now have more programs accredited than they did prior to the first five-year cycle 
of performance funding. Eight of those 12 directly attributed the accreditation of the additional 
programs to Standard I. Five of the eight viewed positively their decisions to have these 
additional programs accredited. On balance, the institutions viewed Standard I in a positive 
way. 

In response to Standard II, which focuses on major field of study, we found that the 
number of institutions using systematic peer review in a comprehensive program review process 
to look . t the quality of programs increased from 3 to 12 during the five-year cycle. At UTK, 
we feel that the peer review process is one of the strongest approaches to assessment; and we 
learned in our survey that other institutions share this view. In our survey, we also found an 
increased percentage of programs using comprehensive exit exams in the major, selecting 
either standardized exams or using locally developed exams. We now have a significant 
percentage of institutions in Tennessee using both standardized exams and locally developed 
exams. Ten of the institutions favored use of Jocal exams, seven favored the use of 
standardized exams, and five indicated no strong preference. At UTK we have tried to give 
flexibility at the departmental level for the faculty to evaluate the strengths and weaknesses cf 
the standardized exams and then examine the potential for developing local exams before 
making decisions concerning which approach makes the most sense for a particular program. 

Institutions in Tennessee have used the results of the standardized exams and/or the 
locally developed exams in the major to identify strengths and weaknesses and suggest 
improvements in curriculum and instructional processes. Program improvements have been 
made at a number of institutions and resulted in better outcomes on the subsequent 
examinations. We found increased attention given to self-study and program evaluation at the 
departmental level. Faculty have been encouraged by high scoies on standardized tests in 
some programs, and in others they have been con irned about lower scores and worked to 
improve student performance. Faculty are asked to consider program objectives, both in 
evaluating the adequacy of standardized exams and in developing new local exams. For the 
first time in some institutions and departments, faculty are looking at the relationship between 
their curriculum and desired program outcomes. We have also seen increased communication 
between faculty and administrators regarding instructional goals. At UTK, we have provided 
seed money to support the development of local exams, giving faculty released time to define 
performance levels appropriate for tk~ students 5 major field of study, and to develop exams 
which accurately measure performance. 
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The survey further indicated that four institutions are testing freshmen using the COMP 
and nine institutions are testing seniors using the COMP to assess general education outcomes. 
The nine institutions which are testing seniors but not using COMP at the freshman level are 
imputing COMP scores using initial ACT scores. There are questions about the validity of this 
approach, but we don't have time to discuss that issue today. Measurement of general 
education outcomes is probably one of the more controversial aspects of our assessment 
program, because ACT-COMP is a standardized test which measures only a portion of the 
general education component. It was the best test that we could find at the time we made the 
decision to test in the general education area. The COMP measured a higher percentage of 
content we wanted to measure in the general education area than did other instruments 
available at that time. 

Our survey indicated that Tennessee institutions have sought to relate performance on 
the COMP to a number of factors: GPA, attrition, types of general education courses taken, 
age, and demographic factors. Careful studies have been made at these institutions in an 
effort to determine relationships between COMP scores and a variety of activities in place at 
the particular institution. 

What has been the result of using COMP throughout the state? We've seen careful 
reviews of the general education curriculum; changes in general education objectives; and 
^rricular changes in areas of problem-solving, the arts, humanities and social sciences. We 
have also seen faculty development initiatives in somr departments to strengthen faculty 
capabilities in areas where departments have not been satisfied with COMP results. 

Probably the most popular single component of all of the performance funding standards 
is the survey activity. Ninety-one percent of the institutions believe the use of surveys is worth 
the cost of the surveying effort. The information coming from surveys of dropouts, alumni and 
enrolled students has encouraged institutions to strengthen advising, give more attention to 
placement of graduates and improve student services, registration and class schedules. We 
have seen a number of efforts to improve student retention using information from dropout 
surveys which reveals differences in responses of dropouts from responses of those who 
persist. We have found factors contributing to attrition which are totally unrelated to academic 
performance that we can give attention to as we seek to improve th* retention of students. 

Improvement of orientation and supervision of graduate teaching assistants, increased 
emphasis on the use of the library, emphasis upon better writing and new faculty development 
initiatives are additional results of the availability of the survey information. 

The fifth standard — planning for instructional improvement — has obviously encouraged 
institutions to incorporate information from a variety of assessment activities to improve 
instructional programs. In 1979, only nine of the 22 institutions were using comprehensive 
planning processes. Now 2i of 22 (95 percent) are using such processes and 50 percent of 
these have said Standard V has influenced their planning by focusing more attention on 
curriculum md program improvement. At our institution, this particular component has led to 
a better focus on certain areas of needed program improvement and enabled us to bring 
together our outcomes assessment and planning activities in a more refined fashion. 
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In summary, major improvements directly related to performance funding have occurred 
throughout higher education in our state. We have increased the efforts of faculty throughout 
the state to monitor and assess student progress. General education curricula have been 
strengthened. Improved planning, improvements in a variety of student services, better 
placement procedures anJ more faculty development initiatives have occurred at a number of 
institutions. We have seen increased community involvement in institutional programs through 
the involvement of emplcyers and community representatives in assessment activities. 
Increased attention has been given to student evaluation of instruction and student services. 
Since the program has now been in place for a number of years, we can look at student 
perceptions of programs and services obtained during the first years of the program and 
compare their responses live years later and clearly understand and appreciate the degree of 
change that has taken place in a number ot key areas. Throughout all of the assessment 
activities, a major contributing factor to the successes that have occurred has been the 
substantial involvement of faculty throughout the process. 
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The Tennessee Experience: The View Fivm a Campus 



Trudy W. Banta 
Research Professor and Director 
Assessment Resource Center 
The University of Tennessee, Knoxville 



In 1983, when it became clear that the Tennessee Higher Education Commission 
(THEC) planned to implement a policy of performance funding, there was some consternation 
a*jung faculty and administrators at the University of Tennessee, Knoxville. This external 
mandate seemed to pose a threat to academic freedom, yet it was difficult to argue with the 
state's contention that public institutions should be held accountable for student learning. 

We felt that we needed to find ways to make the process of outcomes assessment a part 
of the ongoing activities of the university and to involve students and faculty as well as 
administrators in that process. We obtained a small grant from the Kellogg Foundation and 
undertook an initial study to make assessment our own, setting forth some ground rules and 
creating a sense of trust among the members of the campus community. Since then, we have 
shared the results * I assessment with students, faculty, and administrators on an ongoing basis 
because we need the active participation of all these groups to insure that the assessment 
information we collect will be used for improvement. 

Under the aegis of the Kellogg grant, we established three task forces, one to investigate 
methods of assessing student achievement in general education, one to look at achievement in 
the major, and one to study the use of surveys to sample opinion about program quality. These 
task forces meeting as a single group ultimately developed a formal statement of purpose for 
assessment on our campus. It is crucial for any campus to have a single-mindedness about 
why it is undertaking an assessment program. We determined that we would use assessment 
activities to gather information that v ould permit us to strengthen and improve our academic 
programs and student services and that we would incorporate these activities in our ongoing 
comprehensive program review and institutional planning process. 

We utilize an internal planning framework that involves a committee of faculty and 
administrators in considering the external environment of the university, its internal strengths 
and weaknesses, and filtering that through the values and tradition of the institution to make 
recommendations to the central administration about mission, goals and objectives, and 
progress toward achieving our purposes. Following the recommendations of the Kellogg project 
task forces on assessment, the university's planning committee modified the one-paragraph 
statement of mission for the institution to include a sentence about the enforcement of our 
academic standards through a rigorous program of peer review and assessment (see Chart 1). 
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University of Tennessee, Knoxville 

Statement of Mission 
1985 
(An Excerpt) 

. . . The University's high standards are enforced through a rigorous system of 
program review and assessment . . . . 

Rewards for excellence (are) based on regular assessment ... of programs as 
indicated by . . . measures such as the achievement of graduates, student and 
alumni opinion, and professional peer review. 



Prior to 1983, the guidelines for preparing the self-study before a peer review had 
focused on traditional resource measures — that is, the abilities of incoming students, the 
training and level of expertise of faculty, the effectiveness of the department chair, and the 
adequacy of the library collection. As the result of our new interest in assessment, m 1983 we 
modified the self-study guidelines to focus attention on outcomes as well as input indicators. 
Now we are saying to each department that we want to develop a sense of how well their 
students are doing in general education and in their major field, and how program graduates, 
employers, those who have dropped out of the program, and others feel about the instruction 
and assistance being provided by the department. 

As the principal measure of student achievement in general education at UTK, we have 
employed the College Outcome Measures Project (COMP) exam developed by ACT. There 
are two forms of the COMP exam — a six-hour version that gives students an opportunity to 
do some writing and record a speech, and a two-and-one-half hour version that uses a 
multiple-choice response format. Since there is a high correlation (.8) between the longer 
composite exam and the objective test, we administer the shorter test to approximately 1,800 
freshmen and 3,000 seniors annually. 

The COMP exam is described by its developers as a test of "effective adult functioning." 
It tests students' abilities to apply what they have learned in the areas of science* social 
science, and the arts. It also provides subscores in communicating, solving problems, and 
clarifying values. The developers of the COMP have demonstrated that there is a relationship 
between seniors' scores on the test and job supervisors' ratings, participation in civic affairs, 
and the likelihood of continuing one's education after college. We used the COMP exam 
initially with a sample of volunteers in 1980, began testing systematic samples of seniors in 
1983, then made taking the test a requirement for graduation in 1985. We took action to 
require the COMP of all seniors j-.imarily because departments were interested in seeing the 
scores for their own students, and it was impossible to provide this information on the basis of 
a sample of the senior class. 

In providing comprehensive exams in the major for UTK seniors, we utilize 54 different 
commercially available tests and 46 that we have developed on our own. In many of the liberal 
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arts areas, we were able to use the Graduate Record Exam Advanced Tests. But in the 
Colleges of Agriculture, Communications, and Human Ecology, and in the humanities, we 
were forced to develop our own exams because no standardized tests were available. Every 
faculty that developed a comprehensive exam in its discipline did so in consultation with two 
specialists in the field from other institutions. While many of these exams contain n;uitiple- 
choice items constructed by the faculty and validated by the external consultants, most also 
contain some performance components. For instance, in the advertising exam, a client in the 
Knoxville area who has a new product comes to the classroom and explains it to the students. 
Then the seniors develop an advertising campaign which receives a critique from the client. In 
child and family studies, where each senior participates in a field experience, a part of the 
comprehensive examination is a rating by faculty and field supervisors of the senior's 
performance in the field setting. 

Our surveys have proven to be ihe most popular element of oui assessment program. 
Since we decided to develop our own instruments, the surveys had instant credibility with the 
faculty and administrators who were involved in determining tieir content. One distinct 
advantage of constructing your own survey is that you can use the exact titles of programs and 
services on your campus — the "Career Planning and Placement Center" rather than simply 
"the placement service." Since 1983 we have developed a family of related survey instruments 
that contain a number of common items so that we can tell how alumni, enrolled students, and 
graduate students feel about certain aspects of the curriculum and instruction. 

When a department begins the self-study for its program review during the year prior to 
the visit by external consultants, we are in a position to offer the department a variety of 
information about its program outcomes. The department will receive COMP exam scores for 
seniors, scores from its comprehensive exam in the major, results from the annual Student 
Satisfaction Survey for its own students, graduate student survey responses, and information 
derived from a survey of departmental faculty. Pulling together information from a variety of 
sources as we do is far more defensible than any attempt to assess program quality using a 
single measure. 

I would like to emphasize that prior to 1983, many forms of assessment were already 
underway at UiK, just as they are on most other campuses. When our Kellogg task forces 
began to look at what was available to measure program outcomes, they conducted an 
inventory of existing assessment activities, and we have utilized those as part of the overall 
campus assessment program. For instate, many departments were already looking at rates of 
job placement and placement in graduate school, at surveys of graduates and employers, and 
at such assessments of student performance as are provided by portfolios in art, performance 
in dance, case study analyses, and oral presentations. 

I would like to summarize some important assumptic ^ that underlie the University of 
Tennessee's assessment program. We are not focusing on the levels of achievement of 
individual students; rather we are assessing the quality of our academic programs and student 
services by measuring their effectiveness in meeting the goals that faculty and administrators 
have established tor them. No single assessment technique can give us all the information we 
need to effectively evaluate any program. Instead, we must employ multiple measures of 
outcomes. We have attempted to establish trust for the program among faculty and students. 
Thf faculty have been assured that they will not be evaluated for promotion or tenure on the 
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basis of assessment information, and students arc assured that their performance on 
assessment measures will not serve as a barrier to their progress toward a degree. The attitude 
that we would like to convey is that we are all engaged in a process of self-assessmen to find 
out how we can improve. The assessment information is not going to be compiled in a volume 
that sits on a shelf; it is going to be used. We attempt to insure this usage by providing a 
variety of data summaries for various audiences, not only in written form, but in presentations 
that will permit discussion and clarification of issues. 

Now I would like to summarize some of the results of our assessment initiatives. The 
experience of testing UTK seniors in general education has led to additional discussion by 
faculty of good practice in teaching and testing. It has also made the faculty more sensitive to 
the need to set learning outcome objectives for students. Coincidentally, our emphasis on 
assessment occurred almost simultaneously with our decision to change from a quarter to a 
semester calendar, and the discussion of learning outcome objectives fostered by the 
assessment initiative was very helpful as changes in courses and programs for the semester 
system were considered. We used the opportunity provided by the calendar change to employ 
the assessment data to suggest direction for constructing a stronger cars, curriculum in general 
education and to emphasize social science in that core. Interest in a problem-solving-across- 
the-curriculum initiative also was fostered by a concern about student scores on the Solving 
Problems subscale of the COMP exam. 

Testing seniors in their major has stimulated faculty in every discipline to think about 
learning outcome objectives for their own curricula. As a result of these discussions, we 
believe that there is more consistency in the teaching of core courses that are offered in 
multiple sections, and since there is a better SQ^ae of shared purpose with respect to achieving 
curriculum goals, upper and lower division courses are more closely integrated. That is, lower 
division courses actually become prerequisites for upper division work, and there is a more 
conscious effort to build at the upper division on concepts that were taught in lower division 
coursework. After faculty have had an opportunity to study students' scores on the 
comprehensive exams, some have implemented a stronger core curriculum, several have 
provided more structure within certain courses through the use of formal statements of 
objectives, and most have begun to give students more written assignments and more 
opportunities to demonstrate that they can apply what they are learning. 

After reviewing the results of our assessment activities over several years, the central 
administration has made a number of significant changes based on needs identified in the 
assessment process. In some colleges and departments, enrollment limitations have been 
implemented so that class size can be reduced and faculty will ha\e more time for advising 
and working with individual students. A strong emphasis has been placed on advising in all of 
the ten undergraduate colleges— real dollars have been added to strengthen the advising 
function and faculty have been given increased opportunities to acquire released time for that 
purpose. With our survey of graduate students, we found that the most significant area of 
dissatisfaction was the size of stipends available. Accordingly, the administration has made a 
concerted effort to increase graduate students' stipends. 

The university's planning committee annually considers evidence of the achievement of 
uiw institution's goals and related objectives. For the last three years the assessment data has 
been summarized for each goal to which it may be applied (see Chart 2). For instance, the 
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affirmative action goal has a related objective of emphasizing minority retention. Our dropout 
survey reveals that black students are more likely to leave the university for personal and 
financial reasons than are white students. Knowing this has provided the basis for a whole 
series of intervention strategies on the part of our student affairs staff and faculty. 



Chan 2 
UTK Planning Goals 

1. Student body composition 

2. Student development 

3. Academic program 

4. Faculty 

5. Teaching 

6. Research 

7. Affirmative action 

8. Library 

9. Computing/Telecommunications 

10. Economic development 

11. Resource allocation 



I would like to close with some concrete evidence of what can happen when assessmert 
data are collected, carefully analyzed, interpreted for decision-makers, and used to m?ke 
needed improvements. We now have data on student satisfaction for six years and can 
demonstrate that over that time satisfaction has improved with respect to a majority of the 
items on the survey. Responses to some of the items have remained at the same levd, but on 
no item related to instruction or services provided by academic departments has the re been a 
significant decrease over those six years. An example of a dramatic increase is the pattern of 
responses to the item "accuracy of catalog description of courses.' 3 As a result, we relieve, of 
increased agreement among faculty about program objectives, we would expect improvements 
in the congruence between the catalog description and actual course content. Students' 
responses confirm that these improvements have occurred. The lower student/f tculty ratios 
that we have tried to effect at UTK since 1983 have led to increased student satisfaction with 
opportunities for discussion in class and availability of faculty outside class (see Chart 3). 

Six years ago the absence of systematic data made it difficult for us to focus on what we 
ought to be doing to improve the student learning environment at UTK. Today we have a 
series of clear goals that we can distribute to every faculty member. We have achieved 
consensus about those goals and we are motivated to move forward toward achieving them. A 
collective sense of purpose is one of the most positive outcomes of our assessment initiative. 
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Chart 3 

Evidence of Goal Attainment 

Responses Related to Class Size 
(Student Satisfaction Survey) 



Instructor's Availability 
for Consultation 




Quality of CI aw room 
Discus? Ion 
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The King's College Experience 



D. W. Fanner 
Vice President for Academic Affairs 
King's College, Pennsylvania 

I am presenting today a private college perspective on assessment, but what Joseph 
Burke and Trudy Banta have said from a public college perspective points to many 
commonalities. The basic message that needs to be heard is that private and public colleges 
can agree on essential purposes, methods, and data for assessment because they share a 
common goal: to use the assessment process to help students achieve academic success. This 
basic agreement among educators from the public and private sectors provides an important 
message for policymakers. 

Some in higher education wonder if assessment is not just another fad which will soon be 
out of favor. Others outside of higher education see assessment as "medicine" for holding 
higher education accountable. I believe that assessment should be seen in a more positive way 
for the promise it holds to improve both the quality and quantity of learning taking place for 
students. 

If this promise is to be realized, however, the purpose of assessment must first be 
clarified. I view the emerging national debate on assessment to focus directly on the ambiguity 
of purpose. A major gap exists between the concepts of assessment as learning and 
assessment as measuring. Assessment as learning is a faculty-driven diagnostic and formative 
evaluation process aimed at improving student learning by providing continual feedback on 
academic performance to individual students. Assessment viewed as Ci^asunng is an 
administratively driven, standardized, and summative evaluation process designed to produce 
a numerical rating. These two concepts of assessment are not necessarily incompatible. I do 
believe, however, that it matters greatly which one a college chooses as a primary purpose for 
assessing student learning. It is this decision that will determine whether or not a college 
realizes the promise assessment holds for improving higher education. 

Externally mandated assessment programs respond to the administrate and 
information needs of legislators, state boards of education and the public in general. These 
general tests of student learning are usually administered to graduating seniors in order to 
produce a score or grade that can be reported to those mandating these standardized tests. 
Although test and measurement experts may be pleased with the objectivity and validity of 
such an approach to assessment, they do not address the more important educational 
questions concerning the value of the learning being assessed and whether the assessment 
process benefits students. 

If faculty approach assessment as being divorced from learning and as simply being a 
bureaucratic hurdle for students to overcome, assessment will lose its credibility and be treated 
jy faculty and students as busywork. Assessment must make sense to faculty in order for it to 
have an impact on education. Assessments should not be designed to assess what is easiest to 
assess but rather what is more difficult to assess. It is the learning that is mere difficult to 
assess that rc r *esents the real measure of quality in higher education. It is this learning that is 
more appropriately associated with the intended outcomes of liberal education. 
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Colleges need to design assessment strategies to assess what is higher in higher 
education — the ability of students to apply what they have learned to a new stimulus — rather 
than what is lower — the ability to demonstrate information-recall. Memorizing and 
emphasizing information-recall should not be substituted for thinking and applying one's 
understanding to new challenges as the primary goal of higher education. If "assessment" is 
truly the code word for the quality issue -in higher education today, let us be clear about what 
constitutes quality undergraduate education. 

King's College focuses primarily on individual student assessment rather than on 
program assessment. The college is committed to the concept of assessment as learning. Its 
chief characteristic is to provide meaningful feedback to students so that they can then act 
upon that information to improve their performance Assessment at King's College is an 
integral part of the teaching/learning process and i*~s evolved naturally out of the faculty's 
implementation of an outcomes-oriented curriculum. Helping students to become more 
successful learners has contributed to increasing the college's retention rate and that is 
important for a tuition-dependent college. 

Earlier this year, I published a book describing the development of the King's curriculum 
and its course-embedded assessment model. In the preface to this work, Enhancing Student 
Learning: Emphasizing Essential Competencies in Academic Programs, Peter Ewell 
emphasized that excellence at King's College is the fruit of its strategic planning process and 
its willingness to spend time-on-task during the past ten years: "Several themes of the King's 
College experience are particularly noteworthy in the light of national experience. One is the 
often overlooked proposition that excellence begins with action. Visible throughout the 
experiences described in this monograph is a demonstration that excellence in undergraduate 
instruction does not just happen; rather it must be achieved through the intelligent deployment 
of limited resources in the pursuit of carefully defined ends." 

I am frequently asked the question, who or what made you do it? The answer is almost 
inconceivable to most who ask the question: "We did it because we wanted to improve the 
effectiveness of education for students at King's College." In the context of a centralized 
planning process, I formulated objectives requiring stretching for both faculty and students as 
well as a certain degree of institutional risk-taking. It is this kind of planning objective that 
results in change and improvement. If planning is safe and objectives easy to meet, then a 
college is not in a strategic planning mode. The King's faculty responded to the challenge by 
rethinking what it is that students should know and be able to do as educated persons living 
and working in the 21st century. 

My own personal starting point in thinking through an outcomes-oriented curriculum 
occurred se\w*al years earlier when a graduating senic: paused at my office door befo^ 
leaving and said, "Before I graduate there's something you ought to know. There's a lot more 
teaching going on here than learning." This is painful for a faculty member to hear, but at the 
same time is vital for challenging the emphasis faculty place on the teaching part of the 
teaching/learning equation. Learning does not automatically follow simply because a faculty 
member teaches. Looking at the relationship between teaching and learning is crucial. Such an 
examination leads naturally to an even more vital assessment question: How do we actually 
know the quality and quantity of learning that has taken place for students? 
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It was clear to me that King's College faculty needed to develop a conceptual and 
thoroughly academic approach to curriculum development and not the traditional political 
approach based upon departmental turf questions. Acting on the belief that many good 
educational ideas have failed because they were introduced prematurely, I delayed 
consideration of any possible curriculum revision. Rather than rushing into curriculum change, 
I encouraged the King's faculty to spend five fuU years engaged in faculty development 
programs which included "Writing Across the Curriculum," "Critical Thinking Across the 
Curriculum," and a variety of other transferable skills of liberal learning. Faculty at King's 
had become accustomed to talking regularly at coffee about the sad stite of affairs in which 
students could neither write nor apply other academic skills satisfactorily in their courses. 
Those who complained the loudest always implied that other faculty ought to be doing a better 
job. Problems have a much greater chance of being solved when a faculty member says "I 
must do something** rather than "something must be done." The faculty development programs 
instituted at King's College helped faculty members to define the problem more clearly and to 
understand that the transferable skills of liberal learning would never be mastered by students 
until faculty in all disciplines took responsibility for helping students to further develop and to 
apply these skills across the curriculum. 

The King's faculty also began simultaneously to examine ways of encouraging students to 
become more involved in their own learning and to become active rather than passive 
learners. Faculty attended national workshops on student-centered teaching/learning strategies 
and increasingly experimented with these strategies in appropriate units of their courses. 
Faculty members also began to reflect on the relationship of teaching and testing as a means 
of making testing an integral part of the learning process for students. Examination questions 
requiring high-order reasoning skills by students gradually replaced questions designed merely 
for information recall. 

Only after completion of these faculty development activities was the King's faculty 
prepared to entertain the notion of curriculum revision as an opportunity to reconceptualize 
learning. Faculty project teams were created for each anticipated area of learning in a new 
curriculum. These project teams first focused upon the desired student learning citcomes and 
only after reaching this consensus moved on to the question of course content. This orientation 
in faculty thinking placed an emphasis on what students needed to learn rather than on what 
faculty wanted to teach. In the discussions ol desired learning outcomes, faculty drew upon 
their experiences in the faculty development programs to recognize that not all teaching and 
learning strategies foster the desired learning outcomes. Can students actually develop the 
capacity for critical thinking or higher-order reasoning if they passively listen to a lecture each 
day? The result of this approach to curriculum reform was the development of an outcomes 
curriculum consciously linking appropriate pedagogy to specific learning outcomes. 

In a talent-development model, what a student takes away from college is the importar** 
factor in judging excellence. The decision to define excellence at King's College in terms of 
actual student learning outcomes not only requires linking student-centered learning strategies 
with curriculum, but also on documenting that learning has taken place for students at the 
level of publicly established standards and assessment criteria. The King's College assessment 
model provides not only meaningful feedback to students but also documentation that students 
actually meet faculty expectations. 



The assessment program at King's College is intended to be diagnostic and supportive of 
student learning. The primary purpose of academic assessment at King's College is to provide 
systematic feedback to students on their academic progress toward meeting the expectations of 
faculty throughout dl four years of undergraduate studies. Students are then able to act upon 
this feedback to become more successful learners. There are multiple assessment experiences 
that take place for students from the point of entering King's College to the point of 
graduation. No one assessment experience, viewed in isolation, can ever be considered 
adequate. 

Assessment strategies at King's College are primarily embedded in course work and 
therefore are a natural and integral part of the teaching/learning process rather than being 
external and intrusive. Assessment strategies embedded in course work also address directly 
the question of student motivation. Students take assessment seriously because it counts as 
part of the course grade even though faculty assign to it an additional special purpose for 
assessing specific learning objectives. Since all of these assessment activities are part of course 
work, they are unobtrusive measures and viewed by students as course projects and test 
questions requiring the application of the transferable skills of liberal learning. The King's 
College course-embedded assessment model, informed by the concept of assessment as 
learning, stands in stark contrast to assessment models based upon standardized tests which 
provide students only with a score and may or may not relate to what has been actually taught 
and learned. 

Assessment has not been added on to the King's College curriculum. It is an integral 
part of the plan of learning. The first part of the core curriculum focuses on eight transferable 
skills of liberal learning. Students gain mastery of each skill by following a four-year 
Competence Growth Plan. These plans link learning in the core curriculum with learning in 
each student's major program. These skills reflect traditional liberal learning abilities and new 
techno^gical skills. Faculty in each major program are in the process of designing 
Competence Growth Plans in Critical Thinking, Creative Thinking and Problem Solving 
Strategies, Effective Writing, Effective Oral Communication, Quantitative Analysis, Computer 
Literacy, Library and Information Technologies, and Values Awareness. It is important that 
each set of eight four-year Competence Growth Plans be designed by the faculty in the 
student's major field of study because the exit criteria should not necessarily be the same for 
all students. For example, a math major may indeed need to know how to write a computer 
program while an English major needs only to be a computer user. 

Each Competence Growth Plan is set up in the same way: a competence description or 
learning goals, a specific course-embedded strategy as a means of achieving the competence, 
and specific criteria which faculty use to judge student performance. These criteria help to 
define the competence goal for students in addition to helping them become more focused 
learners with resp**'* 4 to priority learning objectives. Faculty have always had such criteria in 
mind but have rarely shared them so clearly with studerts. The very act of writing such criteria 
also has helped faculty to sharpen their own understanding as well as to become clearer about 
their objectives in the classroom. 

It also makes a big difference for students to know as freshmen that the faculty in their 
major programs have clear expectations for them to transfer liberal learning skills to course 
work in their major discipline. Many freshmen accounting majors initially think of accounting 



-67- 



only in terms of numbers. They are surprised to discover that the accounting faculty expect 
them not only to complete writing assignments but also to continue to improve their writing in 
order to meet the desired exit criteria for senior majors established by that accounting faculty. 
For students majoring in accounting, it is the responsibility of the accounting faculty, not of the 
teachers of freshman writing, to help them to understand what constitutes good writing in the 
accounting profession. This can be done only if students practice writing within the context of 
accounting courses as a four-year developmental process. 

The second section of the curriculum is comprised of interdisciplinary areas of study as 
well as traditional disciplines and methodologies. The focus is on ways of knowing. Faculty 
want students to acquire the body of knowledge that traditionally characterizes an educated 
person, but "they also recognize that most of this content is soon forgotten after graduation. 
What students can really hold onto is how to go about knowing in each of the disciplines. All 
these different disciplinary perspectives and ways of knowing need to be brought together to 
provide interdisciplinary solutions to complex problems in the modern world. 

The third part of the curriculum focuses on exploring values and is organized around the 
theme of Responsible Believing and Acting. Students explore values dilemmas in a series of 
courses and course modules in a variety of disciplines. Formal course work in Responsible 
Believing and Acting is complemented by the values across the curriculum faculty development 
program. The objective of this program is to prepare faculty in all disciplines to examine value 
issues and ethical dilemmas arising naturally out of course work. One of the most challenging 
and unique aspects of focusing on ' alues in the curriculum at King's College has been the 
development of capstone courses in each of the four academic divisions of the college for 
senior-level students. These courses are intended to help students to integrate learning in the 
core curriculum with learning in their major field of study by reflecting on the value-questions 
and ethical dilemmas they are most likely to encounter in their personal and professional lives 
after graduating from college. These courses will be team taught by an ethicist and a faculty 
member whose discipline relates to the subject matter of the course. 

The comprehensive assessment model for enhancing student learning outcomes at Kin^s 
College also contains several other components in addition to the Competence Growth Plans. 
Students encounter their initial assessment experience during the summer prior to entering the 
college This assessment is primarily for advisement purposes and placement of students in 
proper levels of courses. Students are assessed in three skill areas, writing, critical thinking, 
and mathematics, in addition to participating in a speech screening program to identify 
students with speech anxiety. 

During the academic year, students experience pre- and post-assessments in all core 
curriculum courses. These assessments focus on the ability of students to think and to 
communicate in each discipline. In skills courses, the assessments focus on determining 
students' ability to use specific skills effectively. Sophomore-Junioi Diagnostic Projects have 
also been designed in each major program for student majors. The assessment is designed to 
serve as a diagnostic screening device to determine the ability of students to transfer critical 
thinking and effective communication skills dt T eloped through the core curriculum to a 
selected question, case study, or project related to their major field of study. The Sophomore- 
Junior Diagnostic Project provides a safeguard against identifying deficiencies for continuing 
students unly at the senior level when it is too late to help students repair these deficiencies. 



The college also administers the ACT-COMP objective exam to all entering freshmen 
and re-administers a similar instrument to students during their senior year. The value-added 
focus provided by the ACT-COMP assessment helps to validate student learning outcomes for 
those outside King's College who might charge that a course-embedded assessment program is 
too subjective and self-serving to be a valid measure of student learning. Here lies the crux of 
the major issue in assessment today- T s the goal of assessment to provide feedback to students 
in order to redirect and to enhance their learning or is it to generate scores for graduating 
seniors to be reported to those outside a college as evidence that learning has taken place? 

The culminating assessment experience for students at King's College is the Senior- 
Level Integrated Assessment. This assessment is intended to provide an opportunity [ox the 
faculty in a student's major field of study to make a holistic judgment of the student's 
education, especially the ability of the student to integrate the transferable skills of liberal 
learning with learning in his or her major field of study. Mo*t departments have developed or 
revised existing senior seminars as the setting for this comprehensive, course-embedded, 
Senior-Level Integrated Assessment. 

The senior assessment is intended to be a performance-based student experience which 
can be evaluated by all faculty members in the student's major field of study as well as by 
professionals in related fields of employment if appropriate. The assessment experience should 
provide the basis for evaluating the following areas of learning: 

1) command of the knowledge base for the major field of study; 

2) master)' of the methodology of the major discipline; 

3) competence in the transferable skills of liberal learning relating to the departmental 
• ompetence statements and four-year competence growth plans. 

Not ill the desired outcomes of undergraduate education can be assessed prior to 
students' giadur.ting from college. King's College also conducts follow-up studies for alumni. 
All gradua es are surveyed one year after the date of their graduation. Students are asked to 
compare tl e quality of the education they received at King's College with the perceived quality 
of the education of those with whom they are now associated in graduate school, professional 
school, or in the work place. Graduates respond in writing to a series of open-ended questions 
in which they are asked to compare their ability with that of others to write effectively, to 
speak effectively, to think clearly, to solve problerrs, to know how to learn independently, and 
to provide a comparative judgment of the q*- !ity of their academic preparation if enrolled in 
graduate or professional school or their level of performance and prospects for promo: ion if 
currently employed. 

I believe that the development of an outcomes-oriented curriculum and course- 
embedded assessment model have significantly improved both the quality and quantity of 
learning for students at King's College. The course-embedded assessment model makes sense 
to faculty and students because it has evolved out of the college's primary concern with 
teaching and learning. It is not something artificial or tangential to the academic enterprise. 
Assessment at King's College is not simply a score to be reported to the state Board of 
Education. It is an integral part of the curriculum design and provides meaningful feedback for 
students to act upon to realize their academic potential. 



Accountability and the Recognition of 
Education as a Collective Value 

E. Thomas Moran 
Assistant Vice President for Academic Affairs 
SUNY College at Pittsburgh 



Obviously I came with some thoughts on assessment to present to you but, it seems to 
me, there are themes that can and should be gleaned from what has been said here today. 
Therefore, let me explore with you what I think is worth highlighting about *oday's discussion. 
To begin with, there seems to be agreement that there are two orientations to assessment. 
Although not incompatible, they are conceptually distinct. One is external and has as its 
primary focus accountability. The other is internal and has as its primary focus improvement 
of educational practices within an institution. Regardless of the value these two approaches 
may offer, we have identified problems in both of them. These problems need to be stated 
clearly because they are the obstacles that must be overcome if the "assessment movement" is 
to have a meaningful and beneficial influence on higher education. Moxeover, some of these 
problems are potentially so dangerous that they must be a\oided or the widespread adoption 
of assessment activities could undermine the unique nature of American higher education. A 
clear-eyed recognition of these problems, however, should serve as an initial step in 
formulating principles to guide assessment efforts and to insure that they operate to serve the 
ideals of higher education in this society. 

Let me explore further the two orientations to assessment which I've noted. One is 
oriented to external constituencies and is intended largely to demonstrate accountability. 
Higher education cannot explain to the public that we won't or can't tell them what we do with 
$100 billion they invest in this enterprise each year. We have a clear responsibility to attempt 
formally to understand and to communicate what happens to students as a result of the whole 
educational process. It is this responsibility which necessitates that we be responsive to the 
issue of external accountability. Accountability is most usefully understood as consisting of two 
elements. One is basic skills. The other is economic productivity and vitality. State 
policymakers are increasingly interested in the relationship between education and economic 
development. Their interest focuses on two components of primary value — one is a trained 
workforce and the other is technology transfer. We can reasonably determine how well 
students should read and write to satisfy the requirements of basic skills education. Insuring 
that they indeed do so is quite another matter. Incidentally, this issue has become important 
because there have been complaints by business leaders throughout the country that graduates 
of American higher education cannot always engage in basic intellectual functions. The other 
issue, that of economic productivity, is one educators typically don't think of as measurable, 
although economists do. They spend a great deal of time determining the exact contribution 
that education makes to economic productivity, and do a good job of it (for a discussion of 
these issues, see for example the work of Psacharopoulous or Windham). Their findings ought 
to be more routinely integrated into research reports on educational effects. 

The other orientation to assessment involves the use of assessment to improve the 
learning and development of students in the context of higher education. The primary focus 
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here is an internal one. When we started our program at Plattsburgh, people would ask 
"What's new about faculty giving students tests or assessing them— haven't you been doing that 
in colleges for a long time?" Clearly, the answer to that question is yes. However, it may be 
helpful to conceptualize assessment as operating on two levels. The micro level, which includes 
the traditional kind of assessment that occurs between teacher and student, is utilized to make 
judgments about a student's performance, usually with respect to a discrete unit of learning 
and domain of knowledge. The other is a macro level of assessment. In this case assessment is 
stood on its head. It uses aggregate measures of student performance to make judgments, not 
about the individual student, but about the institution, its curriculum, and the educational 
practices of its faculty. To that extent, it examines the impact of the total educational 
experience and endeavors to determine whether or not that experience represents something 
more than the sum of its parts. Essentially this represents a new approach to assessment in 
higher education. It requires a new set of terms, and a new conceptualization that borrows 
from the field of evaluation research, as well as from the disciplines of cognitive learning 
theory and human development. Yet, it is distinguishable from these fields. Assessment is 
establishing its own niche consisting of a growing empirical base, acceptable methods, and 
established purposes. 

Having agreed upon these two approaches to assessment, I would like to identify 
problematic elements of both. A first and perhaps fundamental concern is that, in contrast to 
other sectors of the economy, there are no clear performance standards in higher education. 
Consequently, a single standardized set of measures mandated across the system would be 
counterproductive and dangerous. Because higher learning is a complex, holistic process, the 
outcomes are not easily reducible. As a result, there are considerable problems in determining 
what to measure: it is very difficult to make judgments about wisdom, about creativity, about 
originality — the kinds of qualities that may lead to a future Nobel Laureate. 

This leads me to a second point — which is, that tht time horizons for learning are long. 
William James once said that "the teacher touches eternity because no one knows where his 
influence will end." It raises the question of what the teacher does who excites the 
commitment and intellectual capacity of the student who will 01^ day discover the cure for 
cancer. These qualities are hardly known nor readily measurable at the time a student 
graduates from college. Nevertheless, they are clearly outcomes of the learning process. 

A third factor is that education is interactive. By this I mean it is not simply a question 
of what the institution does to a passive student. We glibly talk about institutional "impact" 
and how we are going to measure it. But, impact is not quite that unidirectional or simplistic. 
Some fascinating research is emerging that points out a rather obvious feature of education, 
that the quality of a student's o* effort also affects the outcomes of the educational process. 
Teachers have always known this, but early assessment efforts tended to ignore it. In recent 
years, Bob Pace at UCLA has focused attention on the quality of student effort as an 
influence on educational outcomes and is devising means by which it can be measured. 
Additionally, student peer group interaction may be of grt~i importance in producing positive 
gains in learning. Zelda Gamson highlighted this theme in her book Liberating Learning when 
sht observed that education is as much a sociological experience as it is a psychological one. 
A study by Philip Treisman, an anthropologist at Berkeley, is instructive here. He attempted 
to examine why c ome culturally distinct groups seemed to do much better in certain subjects 
than other groups. Specifically, the distinction between blacks and Asians in math. He used an 
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ethnographic methodology to examine this question and found that the dngle most important 
factor seemed to be that Asian students treated their pursuit of knowledge in math as a social 
activity* They spent a great deal of time interacting with one another in the common study of 
math, whereas black students treated the educational process (with respect to math at least) 
as a solitary one — not a group process and not an interactive one. Consequently, their 
comparative performance in math suffered. When interventions were employed to change 
these patterns, however, their performance improved dramatically. Apparently, student peer 
group interaction and the quality of student effort, as well as the more frequently studied issue 
of faculty-student interaction, all have an impact on the learning process. Unquestionably, 
these are extremely complex factors to isolate and study. An additional factor that 
complicates the comparative ev ation of educational outcomes is the plethora of intellectual 
perspectives which serve as foundation for various curricula at institutions around the 
country. This plethora of competing ideas is linked to the intellectual vitality and creativity of 
the nation* We don't want to put learning on a Procrustean bed or standardize it to the point 
where we undermine our intellectual vitality as a people simply so that we can conveniently 
evaluate student learning. That's a danger that we have to be most careful to avoid in 
undertaking assessment efforts. 

My fifth point relates to the diversity of institutional missions of colleges and universities. 
Ted Fiske discussed this earlier today, as did Carol Boyer and Joe Burke. The United States, 
in contrast to all other nations, educates 57 percent of the age cohort. Depending on how the 
figure is calculated, the closest competitor nation is still probably under 20 percent of the age 
cohort. Education serves a multiplicity of purposes in this society that it does not serve in 
other countries. For example, we have a long history of values education in our religious 
colleges. We also provide vocational training in our technical schools, and have a tradition of 
self-development in our liberal arts colleges, as well as the instrumental purposes of education 
reflected in our professional schools. It is very difficult to provide standardized measures 
across a system with such a diversity of institutional missions. 

Finally, assessment efforts must also contend with the question of what to measure given 
the range of cvUural and individual differences in this society. What is to be valued? Concern 
for this question is reflected in the debate that raged around William Bennett's report To 
Reclaim a Legacy, describing the traditional canons of literature that he believed should be 
emphasized in the humanities. It was also the basic ^*ist in Allan Bloom's rather meandering 
and often fitful book, The Closing of the American Mind. E. D. Hirsch, in his carefully 
researched and argued treatise on Cultural Literacy (its provocative "list" notwithstanding), 
came at the question from a somewhat difL ent perspective. He pointed out the importance of 
a shared body of commonly understood knowledge if public discourse is to take place at all. 
Fie further examined the way in which individuals who lack that nowledge are closed ^ut of 
effective participation in the life of a society. 

The concluding lesson from these points is that no single or simple yardstick can work to 
make assessment judgments across a system. Moreover, it could be dangerous to attempt to do 
so. Such an effort could promote insidious comparisons that would devitalize intellectual, 
institutional, and cultural diversity. Furthermore, it could lead to a vapid reduction^,; in the 
way we conceptualize education. 




In addition to the problems of what to measure that I've alluded to, there are also a host 
of problems in how we measure the outcomes of education. (For an excellent discussion of this 
issue, see Ernie Pascarella's essay entitled "Are Value-Added Analyses Valuable?") A theme 
woven through the discussion today was that measures are inevitably imperfect. Therefore, we 
are going to have to work at developing more effective approaches to assessment that are at 
once more creative and more meaningful to the educational process than those currently 
available. We also are going lo have to adopt a series of principles for the use of assessment 
results. Trudy Banta mentioned those that exist at her own institution. It seems that all 
institutions seriously engaged in acsessment have some values that accompany the development 
of the program on the campus. The language may vary, but the themes are common. We must 
respect that a curriculum is ultimately an article of faith on the part of a faculty about what 
they can and should provide to students. Assessment, viewed merely as a set of technical 
activities divorced from the values and spirit of an institution, is unquestionably dangerous. 

Despite the problems we've noted, we are also in evident agreement that assessment is 
clearly valuable and that it holds much promise to enhance higher education. In this regard, 
some additional themes I saw emerging today warrant emphasis. Among these is the way we 
value higher education. We have long focused on the benefits of education in this society for 
the individual. We are now beginning to focus on education in terms of collective betterment 
and to relate it to our collective well-being, and that really does represent a shift in 
perspective. Ted Fiske talked about the end of "the ecological frontier," noting that the 
boundless resources of our society traditionally allowed us to have an inefficient educational 
program and that we were now going to have to become more like the Japanese and "live by 
our wits." I hadn't heard it expressed quite that way, but it makes me realize that we are 
coming to a very different sense about education. Historically, it has been treated as 
something that benefited the public through what economists call externalities. In this view, 
the private context dominates. Individuals make educational choices for personal enrichment; 
secondarily these choices of personal investment and consumption have positive ramifications 
for the larger society. We're now asserting that education itself is a public good which requires 
a new conceptualization of investment in education. Having come to this nascent conclusion as 
a society, we can interpret the emergence of assessment as a way by which society can ensure 
that higher education will directly and self-consciously serve public and collective purposes. I 
think the assessment movement, to some extent, reflects this growing awareness within our 
society. 

Another point from Ted Fiske: quoting Ernie Boyer, he mused that "maybe ... the 
academy [is] out of sync with the society supporting it." A number of analysts have pondered 
this possibility for quite some time. I have been at many conferences in recent years where 
observers have talked about a pendulum swing in higher education with respect to institutional 
priorities. These observers contend that there is an increasing concern for teaching, as 
opposed to research. It is becoming especially apparent in certain types of institutions, 
particularly those that historically have had a fundamental purpose in the teaching of 
undergraduates. There's a willingness to reverse a tendency that operated in higher education 
for at least 40 years, a tendency identified by Jencks and Riesman in The Academic 
Revolution, for institutions to mimic those above them in the status pecking order. As a 
consequence, all institutions were becoming like research universities. Yet, that orientation can 
be harmful to the morale of faculty who were attracted tQ certain types of institutions because 
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they preferred to devote their professional efforts primarily to teaching. Increasingly, however, 
V/c are now observing in many institutions a rediscovered touchstone of institutional values. 
ITiese values are based on the positive difference education makes in the lives of students. 
However, once we endorse a touchstone based on such ruttitutional values, it's incumbent 
upon us to devise mea i s whereby we can actually assess and determine the positive impact the 
educational experience is having on students. Assessment is, therefore, a way of 
operationalizing those touchstone values. 

Let me diverge here for a moment. We talk frequently about assessment as something 
that takes place at colleges that were once at the margins— once considered second- or 
perhaps third-rate institutions. By contrast, however, Joe Burke has a theory about those 
institutions at the top cf the institutional pecking order where quality is defined exclusively by 
notions of reputation and resources. He contends that these institutions are now highly 
constrained by those definitions. It is difficult for them to undertake innovations. A great deal 
of the creativity in higher education is now occurring at very different kinds of institutions, 
such as Alverno College and Northeast Missouri State University. Despite lis, it is an ironic 
historical footnote that the term value-added, although it took root in some of these formerly 
less prestigious colleges in the late 1970s and early 1980s and was made famous by the writings 
of Alexander Astin, actually originated with David Riesman at Harvard. The first full-scale 
value-added study was done by Dean Whitia in the Office of Instructional Evaluation there in 
the mid-1970s, and relied extensively on the work of Lawrence Kohlberg and William Perry, 
both scholars at Harvard. 

Another theme that emerged today is that assessment must respect the uniqueness of 
institutions and, consequently, must employ a multiplicity of measures and methods that are 
sensitive to the distinctive competence and character of individual institutions. The point has 
been hammered on over and over. Its acceptance recuires the development of innovative 
measures and experimental evaluation programs. This is something that Carol Boyer and 
Peter Ewell uncovered in their research. It's something highlighted by many speakers today. 
The question of developing innovative measures leads to recognition that a culture around 
assessment will have to grow in each institution which engages in assessment activities. The 
development of valuable and complex forms of assessment is going to take faculty and 
educational researchers a considerable time to bring to fruition. Nevertheless, I think the 
assessment movement has already produced some enduring consequences. As Ted Marchese 
and Pat Kutchings have observed in their cogent writings, institutions in the future will now 
give more concerted and systematic attention to the impact they have on students. But 
developing systems for doing so is going to take resources and time. Perhaps Mae West had 
good advice for us on this score when she said: "Anything worth doing — is worth doing 
slowly." 

Which brings me to a final theme that emerged today, which is that too much external 
pressure can undermine the educationally relevant function of assessment and lead to 
manipulative, useless, and uncreative approaches, as well as invidious institutional 
comparisons. Colleges are not airlines, nor are they hospitals. We have to respond to colleges 
as very different types of organizations. The central element of a college is knowledge — its 
transmission in teaching and its creation in scholarship. These are highly creative and highly 
fragile enterprises that can't be measured in the same way we can measure whether airlines 
lose baggage, land on time — or crash. But assessment, developed carefully and wisely, can 
support the activities that are central to the academy. 

ERJC -74- 8 2 



IV. CONCLUSION 



The assessment movement and the reform of undergraduate education will continue to 
command major attention as higher education moves into the 1990s. Given the increasingly 
tight budgets in New York State, colleges and universities will be seriously challenged to 
improve quality and productivity. The authors of this Special Report have argued that the 
demands for accountability put pressure on higher education to introduce new assessment 
strategies in order to improve the quality of undergraduate education. Four factors lie behind 
the accountability push — the need to maintain a competitive edge in the world's market places, 
the values of democratic society, fiscal accountability, and the need for consumer Protection.. 
These external pressures have led to a reform movement, starting initially with K-12 
education and then encompassing higher education. As a result, a new model of excellence, 
the talent-devdopment model, along with several nev or revitalized approaches to 
assessment, are being implemented on a wider scale than ever before. Section II of this report 
identified these strategies and described several institutions and states who are on the cutting 
edge of reform. Speakers at the Norstar Conference provided current information and insights 
into the assessment activities nationally, in several leading states, and at a number of 
pioneering colleges and universities. 

In summarizing the status of assessment efforts across the nation, Carol Boyer from the 
Education Commission of the States identified five practical guidelines that are worth restating 
as part of this conclusion. She advocated that both state and institutional leaders capitalize on 
existing information, use existing "points of contact," create a visible center for assessment 
activity, experiment with pilot piojects, and use results in identifiable ways as part of the 
planning and budgeting process. She also warned practitioners of certain dangers involved in 
the implementation process, especially the difficulties of communication, the timetables and 
rhythms of the political rnd academic worlds, the "press to test/' and the creation of an 
assessment "train on its own track" adding burdens on colleges for producing separate 
assessment information. 

Homer Fisher and Trudy Banta drew important lessons out of the Tennessee experience 
with performance funding that revealed institutions improving the quality of their 
undergraduate programs, engaging in new assessment activities, receiving additional resource 
allocations tied explicitly to improved performance, and the use by faculty of systematic data 
to clarify program objectives, modify curricula, improve course content, and increase student 
satisfaction with their college experiences. Donald Farmer Culled our attention once again to 
the central responsibility faculty have in viewing assessment as a learning strategy, giving 
students feedback about their competencies in an outcomes-oriented curriculum at a private 
college. Finally, Thomas Moran, in his conference wrap-up, called our attention to 
recognizing the collective value of higher education and to view accountability and assessment 
orientations, not as contradictory or opposing forces, but as complementary and reinforcing 
ways to improve undergraduate education. Moran also warned us of viewing assessment as a 
technical or measurement activity largely divorced from the v^'ies and spirit of an institution 
and faculty commitment to excellence. Perhaps Joseph Burke said it most succinctly in the title 
of his speech, "Academic Assessment: The Right Thing for the Right Reason." As Burke 



states: "The true test of the assessment movement will be whether it can develop and establish 
a new definition of institutional excellence and quality in undergraduate education — a 
definition based on inclusion rather than exclusion, on performance rather than prestige .... 
The question is whether the supporters of assessment will have the courage to replace it with a 
broader view of quality that suits the higher educational needs of our diverse and democratic 
society" (Burke, p. 30). 

In this Special Report, the authors have recounted the salient points of the reform 
movement and have introduced broader models of excellence and approaches to assessment 
so that a diverse student population and the values of a democratic society can be enhanced. 
Again, Burke speaks to the dilemma of American democracy in how it reconciles equality with 
quality. "Some fear our resolve for equality will result in the 'mediocrity of the many/ while 
others feel our push for quality will produce the 'elitism of the few\" But as this report sets 
forth, the dilemmas of accountability and assessment, or equality and quality, or access and 
excellence are not mutually exclusive or derived at the expense of the other. Rather, as 
Thomas Jefferson saw early on, education is an indispensable link between equality and 
quality. The dynamic interaction of one enriches the other. The goals of access and 
excellence, of demonstrating accountability and assessing performance, represent a challenge, 
not a choice, for higher education. America has created a higher education system capable of 
resj^ndin^ to that challenge. By raising fundamental questions about the quality of 
undergraduate education, the current assessment movement has begun a debate that will 
enrich the overall educational enterprise as it moves into the 1990s. 
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